HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
young-geng
EasyLM
Python2023–2024
young-geng / EasyLM

EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

AI & Machine LearningLearning & Reference
Explore on GitHubRead Hacker News discussions
4Distinct people
4Mentions
0Explicit recommenders
2.5KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month4 dated mentions

Observed mentions: 2023-04-05 07:38:56 UTC to 2024-04-24 05:05:15 UTC.

0122023-04-01 00:00:00 UTC to 2023-05-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 commentsApr 20232023-05-01 00:00:00 UTC to 2023-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-06-01 00:00:00 UTC to 2023-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-01 00:00:00 UTC to 2023-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-08-01 00:00:00 UTC to 2023-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-01 00:00:00 UTC to 2023-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-01 00:00:00 UTC to 2023-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsOct 20232023-11-01 00:00:00 UTC to 2023-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-12-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2024-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-01 00:00:00 UTC to 2024-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-03-01 00:00:00 UTC to 2024-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-04-01 00:00:00 UTC to 2024-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsApr 2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 20232023-04-01 00:00:00 UTC2023-05-01 00:00:00 UTC202
May 20232023-05-01 00:00:00 UTC2023-06-01 00:00:00 UTC101
Jun 20232023-06-01 00:00:00 UTC2023-07-01 00:00:00 UTC000
Jul 20232023-07-01 00:00:00 UTC2023-08-01 00:00:00 UTC000
Aug 20232023-08-01 00:00:00 UTC2023-09-01 00:00:00 UTC000
Sep 20232023-09-01 00:00:00 UTC2023-10-01 00:00:00 UTC000
Oct 20232023-10-01 00:00:00 UTC2023-11-01 00:00:00 UTC000
Nov 20232023-11-01 00:00:00 UTC2023-12-01 00:00:00 UTC000
Dec 20232023-12-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
Jan 20242024-01-01 00:00:00 UTC2024-02-01 00:00:00 UTC000
Feb 20242024-02-01 00:00:00 UTC2024-03-01 00:00:00 UTC000
Mar 20242024-03-01 00:00:00 UTC2024-04-01 00:00:00 UTC000
Apr 20242024-04-01 00:00:00 UTC2024-05-01 00:00:00 UTC101

What Hacker News says

4 sources
arjvik·Apr 24, 2024
What are people's thoughts on how this compares to: - EasyLM [1] - Levanter [2] - T5X [3] - and more? [1]: https://github.com/young-geng/EasyLM [2]: https://github.com/stanford-crfm/levanter [3]: https://github.com/google-research/t5x Asking because I have worked extensively on training a large model on a TPU cluster, and started with Levanter, then tried MaxText, and finally ended up on EasyLM. My thoughts are: - Levanter is well intentioned but is unproven and lacking in features. For instance, their sharding is odd in that it requires embedding dimension to be a multiple of the number of devices, so I can't test using a model with embedding dimension 768 on a 512-device pod. Lost confidence in Levanter after finding some glaring correctness bugs (and helping get them fixed). Also, while I'm a huge fan of Equinox's approach, it's sadly
Read the rest
underdeveloped (for instance, there's no way to specify non-default weight initialization strategies without manually doing model surgery to set weights). - MaxText was just very difficult to work with. We felt like we were fighting against it every time we needed to change something because we would be digging through numerous needless layers of abstraction. My favorite was after one long day of debugging, I found a function who's only purpose was to pass its arguments to another function untouched; this function's only purpose was to pass its arguments untouched to a new, third function, that then slightly changed them and passed them to a fourth function that did the work. - EasyLM is, as the name says, easy. But on a deeper dive, the sharding functionality seems to be underdeveloped. What they call "FSDP" is not necessarily true FSDP, it's literally just a certain axis that the JAX mesh is being sharded around that happens to shard some data axes and some model weight axes. I'm still searching for a "perfect" JAX LLM codebase - any pointers?
In Maxtext: A simple, performant and scalable Jax LLM
quickthrower2·May 3, 2023
I am quite new to this, I would like to get it running. Would the process roughly be: 1. Get a machine with decent GPU, probably rent cloud GPU. 2. On that machine download the weights/model/vocab files from https://huggingface.co/openlm-research/open_llama_7b_preview... 3. Install Anaconda. Clone https://github.com/young-geng/EasyLM/. 4. Install EasyLM: conda env create -f scripts/gpu_environment.yml conda activate EasyLM 5. Run this command, as per https://github.com/young-geng/EasyLM/blob/main/docs/llama.md: python -m EasyLM.models.llama.llama_serve \ --mesh_dim='1,1,-1' \ --load_llama_config='13B' \ --load_checkpoint='params::path/to/easylm/llama/checkpoint' \ Am I even close?
In OpenLLaMA: An Open Reproduction of LLaMA
andai·Apr 5, 2023
Vicuna's GitHub says that applying the delta takes 60GB of CPU RAM? Is that what you meant by large swap file? On that note, why is any RAM needed? Can't the files be loaded and diffed chunk by chunk? Edit: The docs for running Koala (a similar model) locally say this (about converting LLaMA to Koala): >To facilitate training very large language models that does not fit into the main memory of a single machine, EasyLM adopt a streaming format of model checkpoint. The streaming checkpointing format is implemented in checkpoint.py. During checkpointing, the StreamingCheckpointer simply flatten a nested state dictionary into a single level dictionary, and stream the key, value pairs to a file one by one using messagepack. Because it streams the tensors one by one, the checkpointer only needs to gather one tensor from the distributed accelera
Read the rest
tors to the main memory at a time, hence saving a lot of memory. https://github.com/young-geng/EasyLM/blob/main/docs/checkpoi... https://github.com/young-geng/EasyLM/blob/main/docs/koala.md Presumably the same technique can be used with Vicuna.
In State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.