GPT2

The published samples from the author's replicated GPT2-1.5M [1] look nothing like the ones from OpenAI's original publication. I really would have loved to see a student with no funding replicating a super large state of the art model, but unfortunately he must either have collected lower-quality data for training, or have failed to replicate every aspect of the model. Almost none of the generated samples have anything to do with the input text, and many of them don't even make any sense. [1] https://github.com/ConnorJL/GPT2/tree/master/samples

Here are some samples generated from this version of the 1.5B parameter model: https://github.com/ConnorJL/GPT2/blob/master/samples/ I'm not particularly impressed with the results. Here are some random samples from OpenAI's version of the model for comparison: https://raw.githubusercontent.com/openai/gpt-2/b5ef71a922efc... Of course, take these with a few grains of salt, these aren't apples-to-apples, etc.