Microsoft just showed how artificial intelligence could find its way into many software applications—by writing code on the fly.
At the Microsoft Build developer conference today, the company’s chief technology officer, Kevin Scott, demonstrated an AI helper for the game Minecraft. The non-player character within the game is powered by the same machine learning technology Microsoft has been testing for auto-generating software code. The feat hints at how recent advances in AI could change personal computing in years to come by replacing interfaces that you tap, type, and click to navigate into interfaces that you simply have a conversation with.
The Minecraft agent responds appropriately to typed commands by converting them into working code behind the scenes using the software API for the game. The AI model that controls the bot was trained on vast amounts of code and natural language text, then shown the API specifications for Minecraft, along with a few usage examples. When a player tells it to “come here,” for instance, the underlying AI model will generate the code needed to have the agent move toward the player. In the demo shown at Build, the bot was also able to perform more complex tasks, like retrieving items and combining them to make something new. And because the model was trained on natural language as well as code, it can even respond to simple questions about how to build things.
While it’s unclear how reliably the system might work outside the demo, similar tricks could be used to make other applications respond to typed or spoken commands.
Microsoft has built an AI coding tool called GitHub Copilot on top of the same technology. It automatically suggests code when a developer starts typing, or in response to the comments added to a piece of code. Scott says Copilot is the first instance of what will likely be a slew of “AI-first” products in the coming years, from Microsoft and others. Code-writing AI “lets you think about doing software development in a different way—so you can express an intention for something that you want to accomplish,” he says.
Scott doesn’t provide specific examples, but this could one day mean a version of Windows that locates a particular document and emails it to a colleague when you ask it to, or an AI-imbued version of Excel that turns a dataset into a chart when you ask. “We’re gonna see lots and lots and lots of big productivity wins for all sorts of routine cognitive work that none of us especially enjoys,” Scott says.
