xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

AI & Machine Learning

Explore on GitHub Read Hacker News discussions

4Distinct people

4Mentions

0Explicit recommenders

10.6KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Hackable and optimized Transformers building blocks, supporting a composable construction.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month4 dated mentions

Observed mentions: 2022-11-24 05:25:27 UTC to 2023-07-17 23:10:32 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Comments
Nov 2022	2022-11-01 00:00:00 UTC	2022-12-01 00:00:00 UTC	1	1
Dec 2022	2022-12-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	1	1
Jan 2023	2023-01-01 00:00:00 UTC	2023-02-01 00:00:00 UTC	0	0
Feb 2023	2023-02-01 00:00:00 UTC	2023-03-01 00:00:00 UTC	1	1
Mar 2023	2023-03-01 00:00:00 UTC	2023-04-01 00:00:00 UTC	0	0
Apr 2023	2023-04-01 00:00:00 UTC	2023-05-01 00:00:00 UTC	0	0
May 2023	2023-05-01 00:00:00 UTC	2023-06-01 00:00:00 UTC	0	0
Jun 2023	2023-06-01 00:00:00 UTC	2023-07-01 00:00:00 UTC	0	0
Jul 2023	2023-07-01 00:00:00 UTC	2023-08-01 00:00:00 UTC	1	1

What Hacker News says

4 sources

hunkins·Jul 17, 2023

This enables V1. V2 is still yet to be integrated into xformers. The team replied saying it should happen this week. See the relevant Github issue here: https://github.com/facebookresearch/xformers/issues/795

In FlashAttention-2, 2x faster than FlashAttention

causalmodels·Feb 24, 2023

I'm going to assume you know how to stand up and manage a distributed training cluster as a simplifying assumption. Note this is an aggressive assumption. You would need to replicate the preprocessing steps. Replicating these steps is going to be tricky as they are not described in detail.Then you would need to implement the model using xformers [1]. Using xformers is going to save you a lot of compute spend. You will need to manually implement the backwards pass to reduce recomputation of expensive activations. The model was trained using 2048 A100 GPUs with 80GBs of VRAM. A single 8 A100 GPU machine from Lambda Cloud costs $12.00/hr [2]. The team from meta used 256 such machines giving you a per day cost of $73,728. It takes 21 days to train this model. The upfront lower bound cost estimate of doing this is [(12.00 * 24) * 21 * 256) = ]

Read the rest

$1,548,288 dollars assuming everything goes smoothly and your model doesn't bite it during training. You may be able to negotiate bulk pricing for these types of workloads. That dollar value is just for the compute resources alone. Given the compute costs required you will probably also want a team composed of ML Ops engineers to monitor the training cluster and research scientists to help you with the preprocessing and model pipelines. [1] https://github.com/facebookresearch/xformers [2] https://lambdalabs.com/service/gpu-cloud edit: these costs are for the 65B parameter model.

In LLaMA: A foundational, 65B-parameter large language model

nl·Dec 6, 2022

In 2016 Transformers didn't exist and the state of the art for neural network based NLP was using LSTMs that had a limit of maybe 100 words at most. With new implementations like xformers[1] and flash attention[2] it is unclear where the length limit is on modern transformer models. Flash Attention can currently scale up to 64,000 tokens on an A100. [1] https://github.com/facebookresearch/xformers/blob/main/HOWTO... [2] https://github.com/HazyResearch/flash-attention

In Turing Machines Are Recurrent Neural Networks (1996)

acidburnNSA·Nov 24, 2022

Awesome. I'm installing on Ubuntu 22.04 right now. Ran into a few errors with the default instructions related to CUDA version mismatches with my nvidia driver. Now I'm trying without conda at all. Made a venv. I upgraded to the latest that Ubuntu provides and then downloaded and installed the appropriate CUDA from [1]. That got me farther. Then ran into the fact that the xformers binaries I had in my earlier attempts is now incompatible with my current drivers and CUDA, so rebuiding that one. I'm in the 30-minute compile, but did the `pip install ninja` as recommended by [2] and it's running on a few of my 32 threads now. Ope! Done in 5 mins. Test info from `python -m xformers.info` looks good. Damn still hitting CUDA out of memory issues. I knew I should have bought a bigger GPU back in 2017. Everyone says I have to downgrade pytorch to

Read the rest

1.12.1 for this to not happen. But oh dang that was compiled with a different cuda, oh groan. Maybe I should get conda to work afterall. `torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 5.93 GiB total capacity; 5.62 GiB already allocated; 15.44 MiB free; 5.67 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF` Guess I better go read those docs... to be continued. [1] https://developer.nvidia.com/cuda-downloads?target_os=Linux&... [2] https://github.com/facebookresearch/xformers

In Stable Diffusion 2.0