TOtosh·Mar 22, 2024Submission
T5: Text-to-Text-Transfer-Transformer
1 points · Open discussiongoogle-research
text-to-text-transfer-transformer
Python2019–2024
google-research / text-to-text-transfer-transformer
text-to-text-transfer-transformer
Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
8Distinct people
8Mentions
0Explicit recommenders
6.6KGitHub stars
What to know before exploring
What it is
Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year8 dated mentions
Observed mentions: 2019-10-25 01:19:32 UTC to 2024-03-22 08:20:36 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
8 sources> Has Google/Alphabet publicly released any of their AI models yet?
You mean NLP field changing models from Google like BERT [1]? or Transformers paper [2]? or T5 model [3] (used by company doing ChatGPT like search currently on the front page on HN)?
1. https://arxiv.org/abs/1810.04805 code+models: https://github.com/google-research/bert
2. https://arxiv.org/abs/2112.04426
3. https://arxiv.org/abs/1910.10683 code+models: https://github.com/google-research/text-to-text-transfer-tra...
In AlphaCode Attention VisualizationT5 was open-sourced on release (up to 11B params): https://github.com/google-research/text-to-text-transfer-tra...
It is also available via Hugging Face transformers.
However, the paper mentions T5-XXL is 4.6B, which doesn't fit any of the checkpoints above, so I'm confused.
In Imagen, a text-to-image diffusion modelI'm not sure whether I'd call 10e10 nearly the size of 10e12, but the dataset is open, so it shouldn't be an issue.
https://github.com/google-research/text-to-text-transfer-tra...
In Google Open-Sources Trillion-Parameter AI Language Model Switch TransformerI have to say that I really doubt this as they've released model weights for their several last few models that have been of nearly this size (and have been trained on the same dataset).
See https://github.com/google-research/text-to-text-transfer-tra...
In Google Open-Sources Trillion-Parameter AI Language Model Switch TransformerWhy is Google's T5 not mentioned?
https://github.com/google-research/text-to-text-transfer-tra...
In Natural language generation: The commercial state of the art in 2020The dataset can be obtained around the web. It's mostly CommonCrawl, Reddit, Toronto Book Corpus, and Wikipedia.
You can find a very comparable corpus open sourced and easy to use on the [T5 repo](https://github.com/google-research/text-to-text-transfer-tra...)
In Giving GPT-3 a Turing TestAlthough those are some great results, I wish I could try it out locally...
https://github.com/google-research/text-to-text-transfer-tra...
It drives me nuts that most of these papers / publications don't have code where I can just run:
> python evaluate_model.py
Still exciting, just annoying that I'd have to set up google cloud to try this out.
In Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline