gpt-3

It implies our models are wrong. Consider that a human adolescence is ~9.46x10^6 minutes and a fast speaking rate is ~200words/minute. That sets an upper bound of 1.9 billion words heard during adolescence. ie: human adults are trained on a corpus of less than 1.9B words. To some extent, more data can offset worse models, but I don't think that's the regieme we're currently in. GPT-3 was trained (on among other languages) 181 billion English words - or about 100 times more words than a human will hear by the time they reach adulthood. How is the human brain able to achieve a higher level of success with 1% of the data? 1. https://github.com/openai/gpt-3/blob/master/dataset_statisti...

For discussion's sake: - BFN reached out to A16Z, Worldcoin, Khosla Ventures largely declined to comment, which would mean that at least one person probably had a bit of runway from at least when the requests for comment were submitted. So yeah, you're probably right. - Going from the github repos for GPT 2 and 3, those may have been hard launches: Feb 14 2019, predating the first press for GPT-2 by a few days (was probably made public Feb 14 though) - https://github.com/openai/gpt-2/commit/c2dae27c1029770cea409... May 28 2020, timed alongside the press news for GPT-3 - https://github.com/openai/gpt-3/commit/12766ba31aa6de490226e... - Would it really have to be a conspiracy? Sounds like only one person would have to target a specific date or date range, and without really giving a reason. One of the things that puts a hole in my own think