IAianbicking·May 18, 2023
A few thoughts:
1. It's interesting, but realistically I'm unlikely to get into the Java code where all the interesting logic is
2. My impression of the code is that all the interesting stuff is in World.java [1] and the many models [2]
3. I don't see obvious prompts in here. I'm guessing they are somewhere, though also built dynamically (maybe mostly through string concatenation). In my experience it's essential to be able to see and understand the prompts, both abstract/template prompts and the instantiated prompts and responses. Also, having done it both ways, I find that keeping prompts separate from code is helpful. They feel different and you iterate on them differently.
4. You should be wanting to get to a place where most of your changes are prompt-driven. That is, you may be changing code, but only because you are refactoring you
Read the rest
r prompts in some qualitative way. With some good DSL you can even support qualitative changes, like changing the chain, or making a prompt multi-stage.
5. My intuition is that an important feature of GPT to these applications is the ability to apply multiple frames to the environment. Which is a fancy way of saying many different prompts that include different system prompts and different purposes. But I can't really find the prompts here so I don't know what's happening.
6. The paper this was based on was really inefficient with calls to GPT, costing according to the paper thousands of dollars to run an in-game three-day trial. I think you're going to want to deal with that right away. You want to move more execution out of the LLM (for instance by having the LLM come up with higher-level instructions that are carried out by simpler algorithms). Also you probably need more chattiness to allow the LLM to indicate what information it wants, instead of providing the LLM with all the subjective information a character has.
Anyway, a few thoughts, good luck!
[1] https://github.com/nickm980/smallville/blob/main/smallville/...
[2] https://github.com/nickm980/smallville/tree/main/smallville/...