OpenAI Five bots fall to professional Dota 2 players, but it's not over yet
AI beats semi-pros but falls down in the big leagues
A group of bots from the OpenAI project has fallen in battle against a team of professional human players of Dota 2, the popular MOBA.
The match was the first of a best-of-three contest, the next two of which will be held against different human teams over the remaining two days of Dota 2's annual tournament, The International.
Earlier this year OpenAI crushed a team of semi-pros (only losing when the viewers were allowed to pick heroes) and last year dominated against pro player Dendi in a one-on-one match.
However, it appears that Brazil's Team paiN, who after all make their living playing the game, is at this point more than a match for OpenAI's incipient Skynet.
Dota 2 is a tactical online battle arena game in which teams of five go head-to-head in a race to take down the enemy's base (the ‘Ancient'), while fighting to prevent the other team doing the same.
Attendees have said that the computer's long-term strategy was lacking compared to Team paiN's, and observed that the bots made several odd plays. These included a few moments of walking in circles, and more spent lingering outside the lair of a powerful boss without engaging. The humans took advantage of the lack of opposition to take down structures protecting the AI's main base.
In total the match lasted 45 minutes. While the AI seemed dominant in the early game - and scored more kills throughout the match - the humans were eventually able to wipe them out and expose the Ancient long enough to destroy it.
Filip Wolski, a member of OpenAI's technical staff, told The Register, "It's disappointing to lose, but I'm happy that we could stand against a top team in such a long match."
The technical stuff
While the humans make their living on Dota 2, and spend a huge amount of time every day perfecting their play, the AI can get many more hours of practice in. OpenAI says that its software can play more games in a single day - equivalent to 180 years of play - than a human can in their lifetime.
This practice is just as important for the bots as it is for humans. At first, the AI had no idea how to play or what to do in Dota 2; it was only through a reinforcement learning technology called Rapid that it slowly learned the rules and objectives.
Rapid means that multiple agents can play a large number of games in parallel. The OpenAI Five bots are then trained using the accumulated knowledge from those games, using a proximal policy optimisation algorithm.
All of that training takes an awful lot of compute power - and we mean a lot. Although OpenAI didn't say how much it used to get ready for The International, we do know it used 1280,000 CPU cores and 256 Nvidia P100 GPUs on Google Cloud when it was preparing for informal matches earlier in the year.
All of that training time should give the AI a massive advantage, and that's not even taking into account its map awareness. Unlike humans, the neural networks running the OpenAI bots get a snapshot of all of the information on the map - including the health, position and items of enemy heroes - once every four frames. Between those frames, it is playing blind.
Human players, on the other hand, need to move their characters around the map to see different parts of it, which are normally covered by a cloud called the Fog of War.
These advantages are potent enough that OpenAI had to artificially increase the bots' reaction times; from a near-perfect 80ms to a more realistic 200ms, to reflect pro players' abilities. Despite that, the bots are still reacting with sub-second timing, which humans struggle to compete with.
Despite these advantages, the bots still lost - albeit narrowly. The manner of the loss highlights a limitation in machine learning, which is that mathematically rendering data into software does not grant strategic knowledge.
"The bots are still very good at moment-to-moment, but they seem bad at macro-level decisions," tweeted Mike Cook, an AI researcher at the University of Falmouth.
Greg Brockman, cofounder and CTO of OpenAI, said, "What we proved today is we're right there at the edge of human ability, and the test is whether we can beat a pro team. We're planning to play other pro teams at The International this week… The test for us is whether we can play at a high enough level to win against a professional team this week."
It's still all to play for today and tomorrow.