ILilaksh·Nov 26, 2023
What about this one? https://github.com/XueFuzhao/OpenMoE
In Ask HN: Why is GPT4 better than the other major LLMs?XueFuzhao
OpenMoE
Python2023–2023
XueFuzhao / OpenMoE
OpenMoE
A family of open-sourced Mixture-of-Experts (MoE) Large Language Models
4Distinct people
6Mentions
0Explicit recommenders
1.7KGitHub stars
What to know before exploring
What it is
A family of open-sourced Mixture-of-Experts (MoE) Large Language Models
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by week6 dated mentions
Observed mentions: 2023-08-21 19:34:04 UTC to 2023-11-26 00:05:31 UTC. Weeks start Monday, UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Aug 21 | 2023-08-21 00:00:00 UTC | 2023-08-28 00:00:00 UTC | 1 | 1 | 0 |
| Aug 28 | 2023-08-28 00:00:00 UTC | 2023-09-04 00:00:00 UTC | 0 | 0 | 0 |
| Sep 4 | 2023-09-04 00:00:00 UTC | 2023-09-11 00:00:00 UTC | 0 | 0 | 0 |
| Sep 11 | 2023-09-11 00:00:00 UTC | 2023-09-18 00:00:00 UTC | 0 | 0 | 0 |
| Sep 18 | 2023-09-18 00:00:00 UTC | 2023-09-25 00:00:00 UTC | 1 | 0 | 1 |
| Sep 25 | 2023-09-25 00:00:00 UTC | 2023-10-02 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2 | 2023-10-02 00:00:00 UTC | 2023-10-09 00:00:00 UTC | 0 | 0 | 0 |
| Oct 9 | 2023-10-09 00:00:00 UTC | 2023-10-16 00:00:00 UTC | 2 | 0 | 2 |
| Oct 16 | 2023-10-16 00:00:00 UTC | 2023-10-23 00:00:00 UTC | 0 | 0 | 0 |
| Oct 23 | 2023-10-23 00:00:00 UTC | 2023-10-30 00:00:00 UTC | 0 | 0 | 0 |
| Oct 30 | 2023-10-30 00:00:00 UTC | 2023-11-06 00:00:00 UTC | 0 | 0 | 0 |
| Nov 6 | 2023-11-06 00:00:00 UTC | 2023-11-13 00:00:00 UTC | 1 | 0 | 1 |
| Nov 13 | 2023-11-13 00:00:00 UTC | 2023-11-20 00:00:00 UTC | 0 | 0 | 0 |
| Nov 20 | 2023-11-20 00:00:00 UTC | 2023-11-27 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
6 sourceshttps://github.com/XueFuzhao/OpenMoE
Check out this open source Mixture of Experts research. Could help a lot with performance of open source models.
In Major outages across ChatGPT and APII think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice.
For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications.
Also see https://laion.ai/
I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback.
In OpenAI is too cheap to beatGoogle have released the models and code for the Switch Transformer from Fedus et al. (2021) under the Apache 2.0 licence. [0]
There's also OpenMoE - an open-source effort to train a mixture of experts model. Currently they've released a model with 8 billion parameters. [1]
[0] https://github.com/google-research/t5x/blob/main/docs/models...
[1] https://github.com/XueFuzhao/OpenMoE
In Mixtures of ExpertsIt makes a lot of sense! In fact there's a number of open source projects working on just such a model right now. Here's a great example: https://github.com/XueFuzhao/OpenMoE/
In A hacker's guide to language models [video]OpenMoE – A family of open-sourced Mixture-of-Experts (MoE) LLMs
3 points · Open discussion