Machine Learning - First Steps

Physics-based dogs learn how to run using a genetic algorithm!

Comment 4

GIFs!!

http://i.imgur.com/aPoRP5A.gifv http://i.imgur.com/uk81En5.gifv http://i.imgur.com/jxZ7sZQ.gifv

Presentation: https://docs.google.com/presentation/d/1uRFV-Djx5rJawTdk-jRuddg3tr48bpYid7b2VJb78BE/edit?usp=sharing

Inspiration

Inspired by videos like this about machine learning, and our fascination with how animals know how to move. https://www.youtube.com/watch?v=pgaEE27nsQw

What is it?

We created a physically simulated quadruped that uses machine learning to move in a 3d environment. The quadruped, which is modeled after a canine, uses a custom genetic algorithm to learn to walk. We simulated their muscles and evaluated the dogs using a fitness formula, which determines the best dogs that should start the next generation. Each dog's purpose in life (as evaluated by the fitness score) is to walk as far as possible, with bonus points for standing up and keeping their head up high.

The dogs have instincts, which are rules to speed up the evolutionary process. Their primary instinct is to move the limbs in a random sinusoid pattern, not completely sporadically. They also have an optional instinct to move left and right muscles in sync. The physicality of the dogs includes many physics based bones and joints, which are stimulated by the dog’s brain to move randomly. No premade animations were used and all walking patterns are emergent.

Use-cases

There are many possible future directions to explore with machine learning. These techniques can be used to make learning robots, optimize processes, and improve self driving vehicles.

Future ideas

If this project were continued, we would like to have our creatures learn to respond to variables such as seeking out food sources, avoiding obstacles, or adapting to changes in the simulated physical environment. We also had more animals planned than we had time to model but the code works for any animal, even a human or a fish. In the future, we would like to explore different gaits such as trotting or running. Motions such as sitting, standing up, and jumping may require more thorough biomechanical modeling. In the future, our models might have a flexible spine, muscles, and tendons in order to produce more lifelike simulations.

Share this project:

Updates

posted an update —

More about the algorithm: It's a genetic algorithm with fitness score, this is a good explanation: https://en.wikipedia.org/wiki/Genetic_algorithm

What it means is each dog has muscles and each muscle has values for how it moves its legs. The muscle values are its genotype. Muscle values drive the joint torque based off of a sin wave. Every round we start with the previous best dogs and clone them and mutate the values, adding variations.

public void mutate(float power = 1f){ offset+= power * Random.Range(-0.1f, 0.1f); frequency+= power * Random.Range(-0.1f, 0.1f); amplitude+= power * Random.Range(-0.1f, 0.1f); center+= power * Random.Range(-0.01f, 0.01f); ...

Dogs run for a set amount of time and are evaluated by a fitness function. Here's what we used to make them run forward:

public float getScore(float wD, float wH, float wR){ scoreRecieved = wD*transform.position.z/10f + wH*(transform.position.y - startingHeight)/startingHeight + wR*Vector3.Dot(transform.up, Vector3.up); return scoreRecieved; }

where the arguments are for a weighted value (the most weight is given to the distance traveled and the others are a "bonus"). Best scores are the next dogs to start with, and repeat. The fitness function is the most important part of the simulation. Changing the weights or even changing what is evaluates will be what creates the emergent behaviour we are looking for. Genetic algorithms like this are definitely the least complex kind of machine learning, but a great thing to do in a limited time for a hackathon. And we got the results we were wanting for after only a few hundred generations and got better from there.

Log in or sign up for Devpost to join the conversation.