gpt-3

Can you please explain to me (I don't understand it) why do you think GPT-3 is closed? Yes, they won't share the trained model, but they're sharing the research here[0][1] so you can reproduce easily, aren't they? As I understand it now, it's very fair - training the model is a separate thing from doing (and sharing) the research, is very costly, and would not happen if they were forced to open that too - I also don't understand why should they be. [0] https://arxiv.org/abs/2005.14165 [1] https://github.com/openai/gpt-3

This isn't strictly true. This is comparing to "GPT3 as a few-shot learner"[1] as opposed to the fine tuned models that everyone else use. Few-shot GPT3 outperforms a BERT-based baseline. [1] https://github.com/openai/gpt-3

That’s what it looks like. There’s a comment in this issue calling that out also. [0] Considering how inaccessible it is I’m curious why there’s so much hype around it. Just the cost/benefit of cloud compute resources needed to run the model looks really bad. All of these hypothetical use cases being discussed in other threads would be financially unsustainable. Granted, it’s still yet to be seen how the smaller models will perform. Looking forward to experimenting with those once they’re released! [0] https://github.com/openai/gpt-3/issues/1#issuecomment-636008...

The vast majority (>93%) is English (by document): https://github.com/openai/gpt-3/blob/master/dataset_statisti...

> For GPT-2, their repository is archived Fun fact: the GPT-3 repo (https://github.com/openai/gpt-3 ) is archived too, but it does use the GitHub archive feature, unlike the GPT-2 repo.

With the recent public advances in NLP[0], I can't help but wonder if/when it will become a common suspicion that people are arguing with advanced bots. It doesn't take an expert to see how these sophisticated language models could be used to broadly target narratives online with counter-narratives through sock puppet accounts. Imagine being in control of a fleet of a million social media bots, all with distinct ways of writing and wording their arguments, all with the capacity to engage content that passes through a sentiment analysis and classification filter. A million social media bots, which could all push, say, pro-life, or anti-gun narratives, generating massive amounts of coherent text on the subject. They can engage with dissenters automatically, endlessly, with a demoralizing amount of content, using multiple accounts to reinfor

Since the demos on this page use zero-shot learning and the used model has a 2020-05-03 timestamp, that implies this API is using some form of GPT-3: https://news.ycombinator.com/item?id=23345379 (EDIT: the accompanying blog post confirms that: https://openai.com/blog/openai-api/ ) Recently, OpenAI set the GPT-3 GitHub repo to read-only: https://github.com/openai/gpt-3 Taken together, this seems to imply that GPT-3 was more intended for a SaaS such as this, and it's less likely that it will be open-sourced like GPT-2 was.

Is there a fast.ai like library that allows a novice to try GPT-3? https://github.com/openai/gpt-3 only contains dataset

The models have not yet been released, and it looks like someone has already asked about it in the issues: https://github.com/openai/gpt-3/issues/1