SpaceInvaders.AI

Inspiration

A few months ago, we started to become very interested in AI and the applications of Machine Learning. We knew that for Hack the North, we wanted to do something related to Machine Learning. Although we spent long hours brainstorming ideas, we couldn't agree on one particular project. Fortunately, we stumbled upon a video showcasing an AI playing Super Mario Bros. better than most humans can. This inspired us to pursue developing an AI that could perform well by human standards on classic Atari games. On the day of the hackathon, we looked through some games that seemed challenging and eventually decided on Space Invaders.

What it does

Our AI is able to play Space Invaders fairly well when compared to an average human player. It uses reinforcement learning algorithms, specifically Q-learning to learn the rules and intricacies of the game. The AI was never given explicit commands or directions in which to play the game. It was only given raw pixel values which it took and taught itself the mechanics of the game.

How we built it

There are two main components to out AI. The first being the convolutional neural network (CNN) that the raw pixel data was fed into and processed to identify patterns. During gameplay, it takes in the pixel information to determine what the next best course of action is for the AI. The CNN is written in Python using TensorFlow as it is a powerful tool for constructing complex network architectures. The other component is the Q-learning algorithm that is used to update the rewards system fed to the AI. Together, the underlying algorithm can be stripped away and used for general purpose reinforcement based learning for Atari games.

Challenges we ran into

We used the Space Invaders environment included in the Open AI gym toolkit to render and retrieve the raw pixel data from the frame. We didn't find out until it was too late that Open AI's gym does not support Windows. We spent hours trying to fix the problem but we could not resolve it. We ended up having to process all the raw pixel data on the MacBook that we had instead of splitting up the 1workload. This brings us to the second major challenge that we faced.

As we had to process all of the raw pixel data on one MacBook, we knew that the time it would take longer to train the neural network than we had originally anticipated. So, we had to carefully devise a strategy and endure hours of testing (coupled with sleep deprivation) to make sure that when we do leave the training to complete, that it would do so correctly.

Accomplishments that we're proud of

IT WORKS!!!!!! Given the hurdles that we had to overcome, we are very proud that we were able to get it working as well as it is in the time that we had.

What we learned

Linux based systems are the best for developing! Also, we learned a lot about reinforcement learning through the research that we completed for this project.

What's next for SpaceInvaders.AI

Of course, further optimizations are in order for our AI's network. In addition, since the AI was made to be a general purpose Atari game AI, we will be applying the underlying architecture to other Atari games.