textgenrnn

textgenrnn (https://github.com/minimaxir/textgenrnn) uses a simple Attention Weighted Average at the end of the model for text generation, which in my testing allows the model to learn much better.

As someone who has spent a lot of time working with text-generating neural networks (https://github.com/minimaxir/textgenrnn), I have a few quick comments. 1) The input dataset from Memegenerator is a bit weird. More importantly, it does not distinctly identify top and bottom texts (some have a capital letter to signifify the start of the bottom text, which isn't always true). A good technique when encoding text for these types of things is to use a control token (e.g. a newline) to indicate these types of behaviors. (the conclusion notes this problem: "One example would be to train on a dataset that includes the break point in the text between upper and lower for the image. These were chosen manually here and are important for the humor impact of the meme.") 2) The use of GLoVe embeddings don't make as much sense here, even as a base. Ge

Two of my projects, textgenrnn (https://github.com/minimaxir/textgenrnn) and reactionrnn (https://github.com/minimaxir/reactionrnn) are among the pretrained models. While this is allowed by the MIT License (and there is sufficient attribution to the source repos), it might be helpful to more explicitly state that the curated models are forks/modifications. These models also have a dependency on the source packages, which I can't promise that it won't have breaking changes if I do decide to update the package. I do like the new accompanying examples for those projects, and it's good to see the projects actually being used! :)