Games have a deep history of spurring development in machine learning. Games provide a simple yet aesthetically pleasing environment to explore concepts in machine learning, allowing researchers to focus on developing machine learning concepts. In addition, games provide a diverse set of goals and challenges, giving researchers the opportunity to explore where and why algorithms break down and explore potential fixes. In short, games act as a proving ground for machine learning systems.
Pictured is perhaps the most notable example of deep learning being applied to games. In 2016, DeepMind’s AlphaGo system beat the world Go champion Lee Sedol in 4 out of 5 games played. The shock victory demonstrated how far deep learning had advanced and spurred further research into deep reinforcement learning. DeepMind went on to develop other game-playing systems, such as AlphaStar, which plays popular real-time strategy game Starcraft. More recently, similar systems developed by DeepMind have had more practical uses, such as predicting protein folding patterns and novel crystal structures.
For my project, I decided to explore some of the deep learning techniques used in games. To do so, I decided to focus on the classic arcade game Snake. In Snake, the player controls the movement of the snake to guide it towards fruit. When a fruit is eaten, the snake grows in length. The game ends when the head of the snake collides with its body or the walls. The player’s final score is given by the amount of fruit eaten.
I chose Snake because it is easy to explain and implement. This allowed me to focus on the reinforcement learning aspect of the project, with which I have limited experience. Snake also has several advantages over other games. For one, there is no randomness in Snake and only one player. Furthermore, the player has access to the complete game state when playing Snake, allowing me to just pass the entire game state to the agent/actor.
I chose to use the Q-learning algorithm to teach a neural network to play snake. This is the same algorithm that was used by AlphaGo, and it has become a standard of deep reinforcement learning. I therefore figured it would be a good first reinforcement learning algorithm to learn. The algorithm is also very general, which will allow me to repurpose the code I develop for future projects.
We use cookies to analyze website traffic and optimize your website experience. By accepting our use of cookies, your data will be aggregated with all other user data.