This AI used GPT-4 to become an expert Minecraft player

AI researchers have built a Minecraft bot that can explore and expand its capabilities in the game’s open world – but unlike other bots, this one has essentially written its own code through trial and error and a lot GPT-4 requests.

Called Voyager, this experimental system is an example of an “embodied agent,” an AI that can move and act freely and purposefully in a simulated or real environment. AIs and personal assistant type chatbots don’t have to do things, let alone navigate a complex world to do those things. But that’s exactly what a home robot could do in the future, so there’s a lot of research into how they might do it.

Minecraft is a good place to test such things because it’s a very (very) rough representation of the real world, with simple and straightforward rules and physics, but also complex and open-ended enough that there’s a lot to accomplish. or to try. Purpose-built simulators are also great, but they have their own limitations.

MineDojo is a simulation framework built around Minecraft, because you can’t just plant some random AI there and expect it to figure out what all those blocks and pigs are doing. Its creators (lots of overlap with the Voyager team) have collected in-game YouTube videos, transcripts, wiki articles, and numerous Reddit posts from r/minecraft, among other data, so users can create or refine a AI model. on them. It also allows these models to be assessed more or less objectively by seeing how well they do things like build a fence around a llama or find and mine a diamond.

Voyager excels at these tasks, performing much better than the only other model that comes close, AutoGPT. But they have a similar approach: using GPT-4 to write their own code as they go.

Normally you just have to train a model on all that good Minecraft data and hope it figured out how to fight skeletons when the sun goes down. Voyager, however, starts out relatively naive, and when he encounters things in the game, he has a little internal conversation with GPT-4 about what to do and how.

For example, let’s say night falls and these skeletons come out. The agent has a general idea of ​​this, but he wonders, what would a good player of this game do when there are monsters nearby? Well, says GPT-4, if you want to explore the world safely, you’ll want to craft and equip a sword, then strike the skeleton with it while avoiding being hit. And that general sense of what to do translates into actionable goals: collect stone and wood, build a sword at the crafting table, equip it, and battle a skeleton.

Once these things are done, they are entered into a library of general skills so that later, when the task is “to go deep into a cave to find iron ore”, she does not have to relearn how to fight from scratch. He still uses GPT, but the cheaper and faster GPT-3.5, which tells him which skills are most relevant for a given situation – so he’s not trying to mine the skeleton and fight the ore.

It’s similar to an agent like AutoGPT, faced with an interface it does not yet know, must learn to navigate to achieve its objective. But Minecraft is a much deeper environment than it’s used to solving, so a specialist agent like Voyager does much better. It finds more things, learns more skills and explores a much larger area than other robots.

Interestingly but perhaps not surprisingly, GPT-4 clears the ground with GPT-3.5 (i.e. ChatGPT) when it comes to generating useful code. A test replacing the first with the second caused the agent to hit a wall early on, maybe even literally, and fail to improve. It might not be obvious talking to both models that one is much smarter, but the truth is you don’t have to be particularly smart to carry on a seemingly intelligent conversation (ask me how I know). Coding is much more difficult and GPT-4 was a big update there.

The purpose of this research is not to make Minecraft players obsolete, but to find methods by which relatively simple AI models can improve based on their “experiences”, for lack of a better word. If we want robots to help us in our homes, hospitals and offices, they will have to learn and apply these lessons to future actions.

You can read more about Voyager here.