HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
lucidrains
DALLE-pytorch
Python2021–2023
lucidrains / DALLE-pytorch

DALLE-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

4Distinct people
6Mentions
0Explicit recommenders
5.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 7:27:27 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year6 dated mentions

Observed mentions: 2021-01-08 01:50:12 UTC to 2023-09-04 16:08:16 UTC.

0123452021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC505
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC000
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC101

What Hacker News says

5 sources
What sort of compute do you get access to? There's a lot of cool stuff you could do depending on whether or not you have decent GPU's and for how much time you're allowed to experiment on them. Experimentation is fairly fundamental in practice. There are a lot of pretraining tasks in vision/multimodal that are cool. Largely techniques introduced or refined by OpenAI re-implemented as pytorch open source codebases with varying degrees of success: - Finetune your own CLIP https://github.com/mlfoundations/open_clip - Train a (much smaller) DALLE https://github.com/lucidrains/DALLE-pytorch - Train your own guided diffusion https://colab.research.google.com/drive/1javQRTkALBWLFWnx1K4... (pretty tough, may only be feasible on domain-specific data) - Train a variational autoencoder (VAE) - "VQGAN" from Heidelberg https://github.com/CompVis/tamin
Read the rest
g-transformers - "Discrete VAE", used as the backbone for OpenAI's DALL-E, reimplimented here (and other places) https://github.com/lucidrains/DALLE-pytorch - "VQVAE2" https://github.com/tgisaturday/dalle-lightning
In Ask HN: Computer Vision Project Ideas?
I've been learning about this stuff for about a year now. Your earlier experiments with learning to drive in GTA V were an inspiration for me - because they hit that perfect intersection of machine learning, accessibility in education, and just plain cool. The pandemic hit and Open AI had released DALL-E and CLIP. I was unemployed and bored with my Python skills and decided to just dive in. I found a nice gentleman named Phil Wang on github had been replicating the DALL-E effort and decided to start contributing! You can find that work here https://github.com/lucidrains/DALLE-pytorch and you'll find me here: https://github.com/afiaka87 We have a few checkpoints available with colab notebooks ready and there is also a research team with access to some more compute who will eventually be able to perform a full replication study and match a
Read the rest
similar scale to Open AI and then some because we are also working with another brilliant German team https://github.com/CompVis/ who has provided us with what they are calling a "VQGAN" (if you're not familiar) - which is a variational autoencoder for vision tokens with the neat trick from GAN-land of using a discriminator in order to produce fine details. https://github.com/CompVis/taming-transformers We use their pretrained VQGAN to convert an image into digits. We use another pretrained text tokenizer to convert words to digits. The digits both go into a Transformer architecture and a mask is applied to the image tokens in the transformer so that the text tokens can't see the image tokens. The digits come out and we encode them back into text and image respectively. Then, a perceptual loss is computed. Rinse, wash, repeat. Slowly but surely, text predicts image without ever having been able to actually _see_ the image. Insanity. Anyway, taking a caption and making a neural network output an image from it has again hit that "perfect intersection of machine learning, accessibility in education, and just plain cool". I don't know if you could fit it into the format of your YouTube channel but perhaps it would be a good match?
In GAN Theft Auto [video]

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.