Microsoft Xbox recently introduced Muse, a “generative AI model crafted for gameplay ideation,” with a detailed article on Nature.com and a blog post accompanied by a YouTube video. If you’re puzzled about what “gameplay ideation” entails, Microsoft explains it as the creation of “game visuals, controller actions, or both.” However, Muse doesn’t quite revolutionize the game development process, despite these technological ambitions.
There are still some intriguing aspects to Muse. The model’s training involved intense computational resources, utilizing H100 GPUs. The process required approximately one million updates to stretch a single second of real gameplay into an additional nine seconds of simulated, engine-accurate gameplay. This training primarily drew data from existing multiplayer gameplay sessions.
Instead of just loading up the game on a lone PC, Microsoft orchestrated the training process on a formidable setup of 100 Nvidia H100 GPUs. This setup, while groundbreaking, escalates costs and energy consumption remarkably—and all this to render just nine more seconds of gameplay at a resolution of 300×180 pixels.
Muse’s most fascinating feat showcased was its ability to clone existing in-game props and enemies and maintain their functionality. However, considering the considerable investment in hardware and electricity, one might wonder why traditional development tools weren’t used to simply spawn these elements.
Muse’s capability to mimic object permanence and emulate original game behavior is noteworthy, but when compared to the well-established, efficient video game development processes, its real-world applications seem less impressive.
While we can hold out hope that future iterations of Muse might tackle more complex challenges, for now, it joins a lineup of projects aiming to simulate gameplay entirely through AI. Although glimpses of engine accuracy and object permanence exist, the method feels inefficient for development, testing, or gaming itself. Even after thoroughly analyzing the available information, it’s perplexing to see how Muse would significantly benefit developers or players at this stage.
