llm-consortium

Reminds me of <https://github.com/irthomasthomas/llm-consortium>

You can spin up a version of this at home using simonw's LLM cli with the llm-consortium plugin. Bonus 1: Use any combination of models. Mix n match models from any lab. Bonus 2: Serve your custom consortium on a local API from a single command using the llm-model-gateway plugin and use it in your apps and coding assistants. https://x.com/karpathy/status/1870692546969735361 > uv tool install llm llm install llm-consortium llm consortium save gthink-n5 -m gemini-pro -n 5 --arbiter gemini-flash --confidence-threshold 99 --max-iterations 4 llm serve --host 0.0.0.0 curl http://0.0.0.0:8000/v1/chat/completions \ -X POST \ -H "Content-Type: application/json" \ -d '{ "model": "gthink-n5", "messages": [{"role": "user", "content": "find a polynomial algorithm for graph-isomorphism"}] }' You can also build a con

Woah they used parallel reasoning. An idea I opensourced about a month before GDMs first paper on it. Very cool. https://x.com/GoogleDeepMind/status/1947333836594946337 So you might be able to achieve similar performance at home today using llm-consortium https://github.com/irthomasthomas/llm-consortium

Like llm-consortium? But without the model diversity. https://x.com/karpathy/status/1870692546969735361 https://github.com/irthomasthomas/llm-consortium

https://github.com/irthomasthomas/llm-consortium also, you aren't limited to cli. When you save a consortium it creates a model. You can then interact with a consortium as if it where a normal model (albeit slower and higher quality). You can then serve your custom models on an openai endpoint and use them with any chat client that supports custom openai endpoints. The default behaviour is to output just the final synthesis, and this should conform to your user prompt. I recently added the ability to continue conversations with a consortium. In this case it only includes your user prompt and final synthesis in the conversation, so it mimics a normal chat, unlike running multiple iterations in the consortium, where full iteration history and arbiter responses are included. UV tool install llm llm install llm-consortium llm install llm-mode

I'd just love to see one of the prompt/response pairs demonstrating the technique. Here is an example from my llm-consortium tool. If you want more, you can have them in 5 minutes. https://github.com/irthomasthomas/llm-consortium/blob/main/c...