New AI algorithm can play Atari's Montezuma's Revenge 10 times faster than Google's DeepMind

DeepMind has struggled with Montezuma's Revenge, where players are required to avoid disappearing floors, lasers, fire pits and much more

Researchers from RMIT University in Australia claim to have developed an artificial intelligence (AI) algorithm that can play the classic Atari video game Montezuma's Revenge 10 times quicker than Google's DeepMind.

In 2015, Google's DeepMind published a paper describing their study in which the DeepMind AI was taught to play old Atari video games. The AI was successful in learning the games and demonstrated perfect skills in playing a collection of games, including Enduro, Pong and Breakout.

But, DeepMind AI struggled to master 1980s complex adventure game Montezuma's Revenge, where players are required to avoid disappearing floors, laser gates, fire pits, and many other hurdles to win the game. DeepMind couldn't even collect the first key in that game.

The reason why AI programmes struggle to master adventure games is that the 'rewards' in such games are randomly distributed. The programme can't improve its gameplay until it gets some reward and, then, it struggles to get a reward because it can't improve its gameplay.

Now, RMIT researchers claim their new algorithm can autonomously play Montezuma's Revenge, and do it 10 times quicker than DeepMind's original AI in identifying subgoals, such as jumping over pits and climbing ladders.

The new algorithm learns from its earlier mistakes, and has been developed by combining 'carrot-and-stick' reinforcement learning with an intrinsic motivation approach, which offers rewards to the programme for being curious and autonomously exploring useful sub-goals.

Researchers took inspiration from other video games, such as Pacman and Super Mario, to introduce this different approach. The research work was carried out by associate professor Fabio Zambetta at RMIT, in collaboration with colleagues Professor Michael Dann and John Thangarajah.

"We've shown that the right kind of algorithms can improve results using a smarter approach rather than purely brute forcing a problem end-to-end on very powerful computers," says Professor Zambetta.

Professor Zambetta also believes their work could have potential application outside of gaming, for example, to improve algorithms controlling self-driving cars or robotic assistants, which are required to achieve goals in the real world.

The findings of the study were presented today at the 33rd AAAI Conference on Artificial Intelligence in Hawaii.