AdventureAI

To compensate for this I have put 100 or more terrible text adventure game runs from AI online [0]. And with this post I even link to them directly for the next web crawler. [0] https://github.com/s-macke/AdventureAI/tree/master/storydump

This paper only scratches the surface and feels incomplete, as it references only GPT-4 and mentions appendices that are not included. The examples are two years old. For a more in-depth analysis of chatbots playing text adventures, take a look at my project. I haven’t updated it in a while due to time constraints. [0] https://github.com/s-macke/AdventureAI

I have realized that in my benchmarks [1]. Llama 3 was significantly better than Llama 3.1, which was puzzling. Then I realized, that I changed the provider. And the new one quantized Llama 3.1 with fp8. Then I tried Hyperbolic [2], because they offer the model in different quantizations. As result, Llama 3.1 was better than Llama 3 or at least on par. [1] https://github.com/s-macke/AdventureAI [2] https://app.hyperbolic.xyz/models

Direct link to Llama3's run through [0]. Having played around extensively with Llama 2 70B, these results are extremely impressive and exciting to me. It can do really interesting things like correct itself when it uses the wrong verb or isn't specific enough. Also, its ability to understand and remember the orientation of the rooms shows levels of comprehension that Llama 2 would have completely flopped on: > SITUATION: I've searched the floor in the living room, but found nothing of interest. > THOUGHT: Maybe I should head to the kitchenette and see if I can find any clues there. > COMMAND: Go east [0] https://github.com/s-macke/AdventureAI/blob/master/assets/90...

The transcript is here https://github.com/s-macke/AdventureAI/blob/master/assets/90...

I didn't make text-adventures with LLMs. I try to solve them [0]. So, far, none of the 7 tested models were able to win even one of the easiest text adventures. I tried many prompting techniques. But only GPT-4 was able to play through the first half of the game. [0] https://github.com/s-macke/AdventureAI

In fact, the performance differences between the models are so significant that even a micro benchmark demonstrates their capabilities. For example, consider my analysis [0] based on observing the progression of Large Language Models (LLMs) in a single text adventure. [0] https://github.com/s-macke/AdventureAI#evaluation-of-other-m...