ChatGLM-6B

Wanted to share since I haven't seen much discussion of LLM development in China other than Baidu-Wenxin (at least in US media). Sharing this particular one since it's open-source and apparently started by an undergrad [1]! That said, it seems to fine-tune atop [2] (a pre-trained LLM on Chinese characters). Some other open-source Chinese LLMs I stumbled upon are [3] and [4]. [1] - https://qiyuan-chen.github.io/ [2] - https://github.com/THUDM/ChatGLM-6B/blob/main/README_en.md [3] - https://github.com/LianjiaTech/BELLE [4] - https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/READ...

The ones I saw mentioned so far were Flan, Cerebras, GPT-J, and RWKV. Not yet mentioned: * Pythia https://github.com/EleutherAI/pythia * GLM-130B https://github.com/THUDM/GLM-130B - see also ChatGLM-6B https://github.com/THUDM/ChatGLM-6B * GPT-NeoX-20B https://huggingface.co/EleutherAI/gpt-neox-20b * GeoV-9B https://github.com/geov-ai/geov * BLOOM https://huggingface.co/bigscience/bloom and BLOOMZ https://huggingface.co/bigscience/bloomz

There is ChatGLM[0], a 6 billion parameter Chinese/English bilingual model that is gaining a reputation as the leading locally runnable LLM for code generation. Maybe look into that. Demo is here[1]. [0]https://github.com/THUDM/ChatGLM-6B/blob/main/README_en.md [1]https://huggingface.co/spaces/multimodalart/ChatGLM-6B

Also today: ChatGLM released by Tsinghua University. I've made a separate submission for it: https://news.ycombinator.com/item?id=35150190 The GitHub page is https://github.com/THUDM/ChatGLM-6B. The GitHub description is all in Chinese, but the model itself can handle English queries on a single consumer GPU well. Considering its size, I'd say the quality of its responses are outstanding.

The project description is in Chinese, so I've used the Google translate URL for this submission. The GitHub page for the project is https://github.com/THUDM/ChatGLM-6B I've tried the model on my own machine, the quality is outstanding for both English and Chinese.