Google’s DeepMind lab has unveiled SIMA — a learning artificial intelligence model that will play games for you. It’s customized to play more like a human than a superhuman artificial intelligence. SIMA, which stands for Scalable Instructable Multiworld Agent, is currently just a research experiment.
Over time, SIMA should be able to learn to play any game, even non-linear and open-world games. Although it is not intended to replace existing game artificial intelligence — it is more like another player that is well suited for team play. It combines natural language communication training with 3D world understanding and image recognition.
«SIMA isn’t trained to win the game; it’s trained to play and do what it’s told»,” says Google DeepMind researcher and SIMA co-leader Tim Harley.
Google worked with eight game developers, including Hello Games, Embracer, Tuxedo Labs, Coffee Stain, and others, to train and test SIMA. The researchers connected SIMA to games such as No Man’s Sky, Teardown, Valheim, and Goat Simulator 3 to teach the AI the basics of gaming. In a blog post, Google says that SIMA does not require a special API to play games or access code.
Harley said the team chose games that focused more on open-ended action than narrative to help SIMA learn general gaming skills. If you’ve played or watched a Goat Simulator walkthrough, you know that the point of the game is to do random, spontaneous things, and the researchers hoped that SIMA would learn that spontaneity.
To do this, the team first created a new environment in the Unity engine where the player had to create sculptures to test their understanding of object manipulation. Google then recorded pairs of human players — one playing and the other giving instructions on what to do next — to record the speech instructions. The players then played on their own to show what led to their actions in the game. All of this was fed back to SIMA so that the model could learn to predict what would happen next on the screen.
SIMA now has about 600 basic skills, such as turning left, climbing a ladder, and opening a menu to use the map. Eventually, SIMA can be tasked with performing more complex functions in the game. Tasks such as finding resources and building a camp are still challenging for the model.
SIMA is not meant to replace AI-based NPCs, but to be a player. SIMA project co-leader Frederic Bess said that it is still too early to say how such AI agents can be useful in games outside of the research sphere.
However, just like AI NPCs, SIMA may eventually learn to speak, but for now, it doesn’t. SIMA is still learning how to play games and adapting to games it hasn’t played before. Google says that with advanced artificial intelligence models, SIMA will eventually be able to perform more complex tasks and become an ideal member of a team of players.