HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
facebookresearch
metaseq
Python2022–2026
facebookresearch / metaseq · Archived

metaseq

Repo for external large-scale work

30Distinct people
31Mentions
0Explicit recommenders
6.5KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Repo for external large-scale work

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 5:46:40 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year31 dated mentions

Observed mentions: 2022-05-03 02:04:25 UTC to 2026-10-05 23:51:10 UTC.

0510152022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 17 mentions; 6 submissions, 11 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 11 mentions; 1 submissions, 10 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC17611
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC11110
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC211
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC000
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC101

From the project README

An excerpt from the repository, fetched Nov 13, 2026.

Read the project’s introduction
Metaseq A codebase for working with Open Pre-trained Transformers, originally forked from fairseq. Community Integrations Using OPT with 🤗 Transformers The OPT 125M--66B models are now available in Hugging Face Transformers. You can access them under the facebook organization on the Hugging Face Hub Using OPT-175B with Alpa The OPT 125M--175B models are now supported in the Alpa project, which enables serving OPT-175B with more flexible parallelisms on older generations of GPUs, such as 40GB A100, V100, T4, M60, etc. Using OPT with Colossal-AI The OPT models are now supported in the Colossal-AI, which helps users to efficiently and quickly deploy OPT models training and inference, reducing large AI model budgets and scaling down the labor cost of learning and deployment. Using OPT with CTranslate2 The OPT 125M--66B models can be executed with CTranslate2, which is a fast inference engine for Transformer models. The project integrates the SmoothQuant technique to allow 8-bit quantization of OPT models. See the usage example to get started. Using OPT with FasterTransformer The OPT models can be served with FasterTransformer, a highly optimized inference framework written and maintained by NVIDIA. We provide instructions to convert OPT checkpoints into FasterTransformer format and a usage example with some benchmark results. Using OPT with DeepSpeed The OPT models can be finetuned using DeepSpeed. See the DeepSpeed-Chat example to get started. Getting Started in Metaseq Follow setup instructions here to get started. Documentation on workflows Training API Background Info Background & relationship to fairseq Chronicles of training OPT-175B Support If you have any questions, bug reports, or feature requests regarding either the codebase or the models released in the projects section, please don't hesitate to post on our Github Issues page. Please remember to follow our Code of Conduct. Contributing We welcome PRs from the community! You can find information about contributing to metaseq in our Contributing document. The Team Metaseq is currently maintained by the CODEOWNERS: Susan Zhang, Naman Goyal, Punit Singh Koura, Moya Chen, Kurt Shuster, David Esiobu, Igor Molybog, Peter Albert, Andrew Poulton, Nikolay Bashlykov, Binh Tang, Uriel Singer, Yuchen Zhang, Armen Aghajanya, Lili Yu, and Adam Polyak. License The majority of metaseq is licensed under the MIT license, however portions of the project are available under separate license terms: Megatron-LM is licensed under the Megatron-LM license
Read the full README on GitHub

What Hacker News says

17 sources
I really enjoyed this interview [1] with Emad Mostaque who I gather is probably the key funding and organization player in that stuff. It remains to be seen exactly how "open" it winds up playing out over time, but they're talking a compelling game and the EleutherAI people seem to be pretty heavily involved, which you probably wouldn't do if you weren't serious. Yandex has also put up a ~100B language model [2]. My old colleagues at Meta have also started nibbling around the edges of opening some of this stuff up [3]. The Meta folks still aren't just handing out the big ones, but they're definitely moving the ball forward. In particular their release of the training logs is a really positive development IMHO as it opens the curtains a bit on the reality of training these things: it's a difficult, error/failure-prone process, there's a lo
Read the rest
t of trial-and-error, restarting from checkpoints, etc. Anything that puts downward pressure on the magical thinking is A-OK in my book. The reality of this stuff is exciting/impressive enough: there's no need to embellish or exaggerate. [1] https://www.youtube.com/watch?v=YQ2QtKcK2dA [2] https://github.com/yandex/YaLM-100B [3] https://github.com/facebookresearch/metaseq
In Beyond Hyperanthropomorphism
I really like this trend of large research group writing more about the difficulties they encountered along the way. Meta had also released a (much more detailed) "Logbook" when they published their 175b models in May: https://github.com/facebookresearch/metaseq/blob/main/projec... For example I somehow expected a Chinese research group with such a proven track record to be showered in compute by the government...
In GLM-130B: An Open Bilingual (English and Chinese) Pre-Trained Language Model

Meta uses Hacker News (and more) in OPT (its version of GPT-3)

Source: Their internal logbook (page 101) https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT1… Here is the full list (if you don't want to open the PDF): - BookCorpus.jsonl - CommonCrawl.jsonl - DM_Mathematics.jsonl - EuroParl.jsonl - Gutenberg_PG-19.json! - HackerNews.jsonl - OpenSubtitles.jsonl - OpenWebText2.jsonl - StackExchange.jsonl - USPTO.jsonl - Wikipedia_en.jsoni - ccnewsv2.jsonl - redditflattened.jsonl - stories.jsonl - EnronEmails.jsonl GPT-3 uses even more than that... It is interesting to see exactly what the large AI models use internally. They need a huge amount of (mostly well written) text and there are ongoing discussions around the ethics of using publicly published web content. Soon, this message will join that model (Hi AI overlords! :) )
3 points · Open discussion
If you haven't used Cloud TPUs in a while, I'd encourage you to try them now with TPU VMs and the latest versions of JAX, PyTorch / XLA, or TensorFlow. We've gotten a lot of positive feedback from customers and TRC users, so we think the overall experience has improved a lot, though there's always more we want to do. People especially seem to find Cloud TPUs easy to use in comparison to alternatives when they are scaling up ML training workloads. Once you have a model running on a single TPU core, it is relatively straightforward from a systems perspective to scale it out to thousands of cores. You still need to work through the ML challenges of scaling, but that is more tractable when you aren't simultaneously struggling with systems-level issues. In particular, you don't need to master a sequence of different networking technologies as
Read the rest
you scale up, and the TPU interconnect is so much faster at scale than other technologies (10X last time I checked) that you don't have to work as hard to avoid network bottlenecks. Support for model parallelism on Cloud TPUs is improving across the ML frameworks, too. To be clear, training ML models at scales that we currently consider large is still very challenging on any platform - for example, the logbooks that Meta recently published are fascinating: https://github.com/facebookresearch/metaseq/blob/main/projec... We aim for Cloud TPUs to simplify the process of training models at these scales and far beyond: https://ai.googleblog.com/2022/04/pathways-language-model-pa...
In Cloud TPU VMs are generally available

Showing the 15 most recent of 17 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.