llm-council

> I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Yes, this technique and its variations[1][2] "work" but it's still not 100% perfect. And it's not as widely used it might be because, among other reason: a. it takes longer to implement b. it costs more (more tokens spread across multiple llm calls) c. higher latency (getting an answer takes longer due to multiple llm calls involved) d. the final answer is probabilistically more likely to be correct, but is still not guaranteed to be error free, so you can never fully escape the need for Human in the Loop. [1]: https://en.wikipedia.org/wiki/LLM-as-a-Judge [2]: https://github.com/karpathy/llm-council

https://github.com/karpathy/llm-council https://ui.adsabs.harvard.edu/abs/2025arXiv250214815C/abstra... https://www.arxiv.org/abs/2509.23537 https://www.aristeidispanos.com/publication/panos2025multiag... https://arxiv.org/abs/2305.14325 https://arxiv.org/abs/2306.05685 https://arxiv.org/abs/2310.19740v1

LLM pattern [0] will probably eventually emerge as the best way to fight those biases. This way everyone benefits from token burn! [0](https://github.com/karpathy/llm-council)

Sounds very similar to LLM council https://github.com/karpathy/llm-council