Thank you for the project! Also sharing the open-source, in-browser, BYOK followup on it
@karpathy, thank you for open-sourcing this repository and for the idea of using councils of LLMs themselves This pattern/ritual of asking multiple LLMs at once with the same question, trying to get wider perspectives and a more solid/grounded answer, was something I caught myself doing periodically.
I did some sort of follow-up on this project and open-sourced the trekhleb/yesbrainer, which is similar in purpose but fully in-browser, bring-your-own-keys kind of app.
I've added the Consencus mode there, which was the funniest, actually. So the models try to come up with arguments for their opinions, try to convince the other models to change their minds, or shift their point of view under the weight of the arguments. This was very exciting to see. For example, in the Demo council: Best Third Language - French over Mandarin, Claude Fable was in the minority at first (suggesting French), but it was able to actually convince GPT and Gemini to switch from suggesting Mandarin to French. The mediator of the Consensus round also highlights the divergence point and logs which models held and which shifted.
On a more general note: thank you for explaining complex topics in such an understandable way (I'm thinking of the YouTube videos on ML). I'm excited to see Eureka Labs evolve, and to find out what shape educational content takes with LLMs.
Apologies for using the issue tracker for a non-issue (there's no Discussions tab here), feel free to close.
Source: karpathy/llm-council