BlenderGPT

Understandable. For context, the GPT in the name comes from an earlier version of this project (https://github.com/gd3kr/blendergpt) which actually used GPT-4 to write python scripts that Blender would then execute. This would allow GPT-4 to program operations like instantiating primitives with the Blender Python API given only a text prompt (ex. "create 50 cubes") The new version of BlenderGPT (lets call this v2) doesn't use an any autoregressive token prediction for the actual mesh generation part, so I understand why it sounds dishonest. I really just chose to stick with the name because artists really didn't seem to care about how the meshes are generated, and the term GPT became closely associated with AI. As for the technical stuff, I've been working on BlenderGPT v2 for the past several months, and until a week ago, i had been usin

Same/different to https://github.com/gd3kr/BlenderGPT ?

I've seen a few interesting ones: https://github.com/gd3kr/BlenderGPT - actually an attempt to make something useful, though its working with meshes and not step files. https://github.com/JennyList/Breakfast-by-ChatGPT - project to get GPT to output OpenSCAD. Mostly just messing around but interesting attempt nonetheless. If I were gonna work on a tool in that space, I'd try to get it running with eg FreeCAD macros or OpenSCAD before trying to get it to work with something like Solidworks. For anything sufficiently complex, a natural language interface is going to be hard though. How do you get it to understand you want it toleranced in Y way vs X way? How do you get it to fix the non-manifold mess it created by destroying all the constraints when you asked it to fillet something? Not impossible certainly, but a lot of work for the librar

I don't know about that. I think it is fully plausible that products like Blender will get LLM integration over the next few years. Already this looks quite useful and it's made by one guy in a week. https://github.com/gd3kr/BlenderGPT

I see a lot of people asking how does this work. The method he is using is one shot learning. He has a prompt and an example of what the interaction should look like. You can see the prompt here: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py It is really easy to build this kind of thing - I've got a very simple command line chatbot that should be very understandable and you can easily play with the prompt. https://github.com/atomic14/command_line_chatgpt I would also recommend that people try out the openai playgrounds. They are great for experimenting with parameters.

He's using the fact the the blender python API is part of the training set. The prompt used can be seen in this file: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py

You can actually just look in this file and see that the first method is being used. He feeds in a prompt with a basic example. https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py

Looking at the underlying prompt [1], it is a one-shot prompt, i.e. it contains one example of a natural language task along with the corresponding Python code (plus a "system" prompt for overall context and direction). Amazing how much you can do with a simple prompt. Imagine Jupyter notebooks with this capability. Or Photoshop. Or Davinci Resolve. We live in amazing times. [1]: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py