HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
facebookresearch
xformers
Python2022–2023
facebookresearch / xformers

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

4Distinct people
4Mentions
0Explicit recommenders
10.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Hackable and optimized Transformers building blocks, supporting a composable construction.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 1:02:30 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month4 dated mentions

Observed mentions: 2022-11-24 05:25:27 UTC to 2023-07-17 23:10:32 UTC.

012022-11-01 00:00:00 UTC to 2022-12-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsNov 20222022-12-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-01-01 00:00:00 UTC to 2023-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-02-01 00:00:00 UTC to 2023-03-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-03-01 00:00:00 UTC to 2023-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsMar 20232023-04-01 00:00:00 UTC to 2023-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-01 00:00:00 UTC to 2023-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-06-01 00:00:00 UTC to 2023-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-01 00:00:00 UTC to 2023-08-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJul 2023

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Nov 20222022-11-01 00:00:00 UTC2022-12-01 00:00:00 UTC101
Dec 20222022-12-01 00:00:00 UTC2023-01-01 00:00:00 UTC101
Jan 20232023-01-01 00:00:00 UTC2023-02-01 00:00:00 UTC000
Feb 20232023-02-01 00:00:00 UTC2023-03-01 00:00:00 UTC101
Mar 20232023-03-01 00:00:00 UTC2023-04-01 00:00:00 UTC000
Apr 20232023-04-01 00:00:00 UTC2023-05-01 00:00:00 UTC000
May 20232023-05-01 00:00:00 UTC2023-06-01 00:00:00 UTC000
Jun 20232023-06-01 00:00:00 UTC2023-07-01 00:00:00 UTC000
Jul 20232023-07-01 00:00:00 UTC2023-08-01 00:00:00 UTC101

What Hacker News says

1 sources
I'm going to assume you know how to stand up and manage a distributed training cluster as a simplifying assumption. Note this is an aggressive assumption. You would need to replicate the preprocessing steps. Replicating these steps is going to be tricky as they are not described in detail.Then you would need to implement the model using xformers [1]. Using xformers is going to save you a lot of compute spend. You will need to manually implement the backwards pass to reduce recomputation of expensive activations. The model was trained using 2048 A100 GPUs with 80GBs of VRAM. A single 8 A100 GPU machine from Lambda Cloud costs $12.00/hr [2]. The team from meta used 256 such machines giving you a per day cost of $73,728. It takes 21 days to train this model. The upfront lower bound cost estimate of doing this is [(12.00 * 24) * 21 * 256) = ]
Read the rest
$1,548,288 dollars assuming everything goes smoothly and your model doesn't bite it during training. You may be able to negotiate bulk pricing for these types of workloads. That dollar value is just for the compute resources alone. Given the compute costs required you will probably also want a team composed of ML Ops engineers to monitor the training cluster and research scientists to help you with the preprocessing and model pipelines. [1] https://github.com/facebookresearch/xformers [2] https://lambdalabs.com/service/gpu-cloud edit: these costs are for the 65B parameter model.
In LLaMA: A foundational, 65B-parameter large language model

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.