HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
openai
gpt-3
AI & Machine Learning2020–2024
openai / gpt-3 · Archived

gpt-3

GPT-3: Language Models are Few-Shot Learners

AI & Machine LearningLearning & Reference
Explore on GitHubRead Hacker News discussions
19Distinct people
22Mentions
0Explicit recommenders
15.7KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

GPT-3: Language Models are Few-Shot Learners

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year22 dated mentions

Observed mentions: 2020-05-29 01:47:27 UTC to 2024-05-13 11:32:35 UTC.

0369122020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 11 mentions; 3 submissions, 8 comments20202021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 7 mentions; 1 submissions, 6 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC1138
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC000
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC202
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC716
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC202

What Hacker News says

22 sources
famouswaffles·Apr 29, 2024
>I wasn't talking about "state of the art LLMs," I am aware that commercial offerings are much better trained in Spanish. This was a thought experiment based on comments from people testing GPT-3.5 with Swahili. A thought experiment from other people comments on another language. So...No. Fabricating failure modes from their personally constructed ideas about how LLMs work seems to be a frustratingly common occurrence in these kinds of discussions. >Frustratingly, just few months ago I read a paper describing how LLMs excessively rely on English-language representations of ideas, but now I can't find it. Most LLMs are trained on English overwhelmingly. GPT-3 had a 92.6% English dataset. https://github.com/openai/gpt-3/blob/master/dataset_statisti... That the models are as proficient as they are is evidence enough of knowledge transfer cl
Read the rest
early happening. https://arxiv.org/abs/2108.13349. If you trained a model on the Catalan tokens GPT-3 was trained on alone, you'd just get a GPT-2 level gibberish model at best. I don't doubt you, i just don't think it means what you think it means. As for papers, these are some interesting ones. How do languages influence each other? Studying cross-lingual data sharing during LLM fine-tuning - https://arxiv.org/pdf/2305.13286 Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer - https://arxiv.org/abs/2404.04042 Zero-shot cross-lingual transfer in instruction tuning of large language models - https://arxiv.org/abs/2402.14778 Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment - https://arxiv.org/abs/2305.05940 It's not like there is perfect transfer but the idea that there's none at all seemed so ridiculous to me (and why i asked the first question). Models would be utterly useless in multilingual settings if that were really the case. Like the 3rd paper, GPT's instruction tuning is done basically only in English - https://mobile.twitter.com/janleike/status/16252072516309606...
In GPT-4.5 or GPT-5 being tested on LMSYS?
seanhunter·Jul 4, 2023
OpenAI publishes a huge amount of research and makes huge amounts of data available for research. They do collaborate with other institutions and researchers. eg: GPT-2: paper https://d4mucfpksywv.cloudfront.net/better-language-models/l... with code: https://github.com/openai/gpt-2, output dataset https://github.com/openai/gpt-2-output-dataset and a bunch of blog posts with all manner of details GPT-3: paper https://arxiv.org/abs/2005.14165 dataset https://github.com/openai/gpt-3 etc etc.
In OpenAI temporarily disables the Browse with Bing beta feature
famouswaffles·Apr 13, 2023
>Certainly it is pleasing that they are not just doing Hindi, but some of these languages must be represented online by a very small corpus of text indeed. I wonder how effectively an LLM can be trained on such a small training set for any given language? as long as it's not the main language it doesn't really matter. Besides English(92.6%), the biggest language by representation (word count) is taken up by french at 1.8%. Most of the languages GPT-3 knows are sitting at <0.2% representation. https://github.com/openai/gpt-3/blob/master/dataset_statisti... Competence in the main language will bleed into the rest.
In BharatGPT: India's Own ChatGPT
famouswaffles·Apr 10, 2023
Doesn't really matter. There's lots of positive transfer in individual language learning. Competence in one language bleeds into competence in others. https://arxiv.org/abs/2108.13349 GPT-3 is fluent in many languages despite English taking up 93% of the corpus by word count. French is next with 1.8% https://github.com/openai/gpt-3/blob/master/dataset_statisti... Dunno the statistics of language presence with GPT-4 but it takes it up another level in terms of its multilingual capabilities.
In Why is GPT-3 15.77x more expensive for certain languages?
kelseyfrog·Apr 11, 2022
It implies our models are wrong. Consider that a human adolescence is ~9.46x10^6 minutes and a fast speaking rate is ~200words/minute. That sets an upper bound of 1.9 billion words heard during adolescence. ie: human adults are trained on a corpus of less than 1.9B words. To some extent, more data can offset worse models, but I don't think that's the regieme we're currently in. GPT-3 was trained (on among other languages) 181 billion English words - or about 100 times more words than a human will hear by the time they reach adulthood. How is the human brain able to achieve a higher level of success with 1% of the data? 1. https://github.com/openai/gpt-3/blob/master/dataset_statisti...
In DeepMind’s New Language Model, Chinchilla
eganist·Apr 6, 2022
For discussion's sake: - BFN reached out to A16Z, Worldcoin, Khosla Ventures largely declined to comment, which would mean that at least one person probably had a bit of runway from at least when the requests for comment were submitted. So yeah, you're probably right. - Going from the github repos for GPT 2 and 3, those may have been hard launches: Feb 14 2019, predating the first press for GPT-2 by a few days (was probably made public Feb 14 though) - https://github.com/openai/gpt-2/commit/c2dae27c1029770cea409... May 28 2020, timed alongside the press news for GPT-3 - https://github.com/openai/gpt-3/commit/12766ba31aa6de490226e... - Would it really have to be a conspiracy? Sounds like only one person would have to target a specific date or date range, and without really giving a reason. One of the things that puts a hole in my own think
Read the rest
ing here is that Sam Altman's name isn't really tied to the Dall-E 2 release. It's just OpenAI, and the press around Sam's name today still exclusively surfaces just this one Worldcoin story (https://news.google.com/search?q=sam+altman+when%3A1d&). So if this was actually intended to bury another story, Sam's name would have to have been included in all the press blasts to be successful. But the Buzzfeed story seems like it kinda died alone on the vine.
In Dall-E 2
emteycz·Oct 7, 2020
Can you please explain to me (I don't understand it) why do you think GPT-3 is closed? Yes, they won't share the trained model, but they're sharing the research here[0][1] so you can reproduce easily, aren't they? As I understand it now, it's very fair - training the model is a separate thing from doing (and sharing) the research, is very costly, and would not happen if they were forced to open that too - I also don't understand why should they be. [0] https://arxiv.org/abs/2005.14165 [1] https://github.com/openai/gpt-3
In AI training method exceeds GPT-3 performance with fewer parameters
peterthehacker·Aug 5, 2020
That’s what it looks like. There’s a comment in this issue calling that out also. [0] Considering how inaccessible it is I’m curious why there’s so much hype around it. Just the cost/benefit of cloud compute resources needed to run the model looks really bad. All of these hypothetical use cases being discussed in other threads would be financially unsustainable. Granted, it’s still yet to be seen how the smaller models will perform. Looking forward to experimenting with those once they’re released! [0] https://github.com/openai/gpt-3/issues/1#issuecomment-636008...
In GPT-3 GH repo archived and Model release remains an open issue

Showing the 15 most recent of 22 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.