taming-transformers

Taming Transformers for High-Resolution Image Synthesis

Creative & Media AI & Machine Learning

Explore on GitHub Read Hacker News discussions

2Distinct people

3Mentions

0Explicit recommenders

6.5KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Taming Transformers for High-Resolution Image Synthesis

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

lucidrains/dalle-pytorch · linked in 2 of the same discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week3 dated mentions

Observed mentions: 2021-07-02 14:44:41 UTC to 2021-10-03 19:39:13 UTC. Weeks start Monday, UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Comments
Jun 28	2021-06-28 00:00:00 UTC	2021-07-05 00:00:00 UTC	2	2
Jul 5	2021-07-05 00:00:00 UTC	2021-07-12 00:00:00 UTC	0	0
Jul 12	2021-07-12 00:00:00 UTC	2021-07-19 00:00:00 UTC	0	0
Jul 19	2021-07-19 00:00:00 UTC	2021-07-26 00:00:00 UTC	0	0
Jul 26	2021-07-26 00:00:00 UTC	2021-08-02 00:00:00 UTC	0	0
Aug 2	2021-08-02 00:00:00 UTC	2021-08-09 00:00:00 UTC	0	0
Aug 9	2021-08-09 00:00:00 UTC	2021-08-16 00:00:00 UTC	0	0
Aug 16	2021-08-16 00:00:00 UTC	2021-08-23 00:00:00 UTC	0	0
Aug 23	2021-08-23 00:00:00 UTC	2021-08-30 00:00:00 UTC	0	0
Aug 30	2021-08-30 00:00:00 UTC	2021-09-06 00:00:00 UTC	0	0
Sep 6	2021-09-06 00:00:00 UTC	2021-09-13 00:00:00 UTC	0	0
Sep 13	2021-09-13 00:00:00 UTC	2021-09-20 00:00:00 UTC	0	0
Sep 20	2021-09-20 00:00:00 UTC	2021-09-27 00:00:00 UTC	0	0
Sep 27	2021-09-27 00:00:00 UTC	2021-10-04 00:00:00 UTC	1	1

What Hacker News says

3 sources

ShamelessC·Oct 3, 2021

What sort of compute do you get access to? There's a lot of cool stuff you could do depending on whether or not you have decent GPU's and for how much time you're allowed to experiment on them. Experimentation is fairly fundamental in practice. There are a lot of pretraining tasks in vision/multimodal that are cool. Largely techniques introduced or refined by OpenAI re-implemented as pytorch open source codebases with varying degrees of success: - Finetune your own CLIP https://github.com/mlfoundations/open_clip - Train a (much smaller) DALLE https://github.com/lucidrains/DALLE-pytorch - Train your own guided diffusion https://colab.research.google.com/drive/1javQRTkALBWLFWnx1K4... (pretty tough, may only be feasible on domain-specific data) - Train a variational autoencoder (VAE) - "VQGAN" from Heidelberg https://github.com/CompVis/tamin

Read the rest

g-transformers - "Discrete VAE", used as the backbone for OpenAI's DALL-E, reimplimented here (and other places) https://github.com/lucidrains/DALLE-pytorch - "VQVAE2" https://github.com/tgisaturday/dalle-lightning

In Ask HN: Computer Vision Project Ideas?

ShamelessC·Jul 2, 2021

I've been learning about this stuff for about a year now. Your earlier experiments with learning to drive in GTA V were an inspiration for me - because they hit that perfect intersection of machine learning, accessibility in education, and just plain cool. The pandemic hit and Open AI had released DALL-E and CLIP. I was unemployed and bored with my Python skills and decided to just dive in. I found a nice gentleman named Phil Wang on github had been replicating the DALL-E effort and decided to start contributing! You can find that work here https://github.com/lucidrains/DALLE-pytorch and you'll find me here: https://github.com/afiaka87 We have a few checkpoints available with colab notebooks ready and there is also a research team with access to some more compute who will eventually be able to perform a full replication study and match a

Read the rest

similar scale to Open AI and then some because we are also working with another brilliant German team https://github.com/CompVis/ who has provided us with what they are calling a "VQGAN" (if you're not familiar) - which is a variational autoencoder for vision tokens with the neat trick from GAN-land of using a discriminator in order to produce fine details. https://github.com/CompVis/taming-transformers We use their pretrained VQGAN to convert an image into digits. We use another pretrained text tokenizer to convert words to digits. The digits both go into a Transformer architecture and a mask is applied to the image tokens in the transformer so that the text tokens can't see the image tokens. The digits come out and we encode them back into text and image respectively. Then, a perceptual loss is computed. Rinse, wash, repeat. Slowly but surely, text predicts image without ever having been able to actually _see_ the image. Insanity. Anyway, taking a caption and making a neural network output an image from it has again hit that "perfect intersection of machine learning, accessibility in education, and just plain cool". I don't know if you could fit it into the format of your YouTube channel but perhaps it would be a good match?

In GAN Theft Auto [video]

adyer07·Jul 2, 2021

It looks like you can - the repo for VQ-GAN has instructions for this: https://github.com/CompVis/taming-transformers#training-on-c...

In Alien Dreams: An Emerging Art Scene