word2gm

> One important thing to note is that Word2Vec does not consider the positional variable of the context words. There is a very interesting new approach of learning Gaussian mixtures from words, which addresses this problem: https://github.com/benathi/word2gm It also solves another interesting problem, which is that Word2Vec doesn't always put "categories" close to their actual contents. Eg. I wrote a simple AI for playing Codenames with word vectors: https://github.com/thomasahle/codenames But where a human use the clue "Countries 2" for "Germany" and "England", the Glove vectors (at least in my implementation) much prefers the clue "France 2", which is pretty confusing at first.

>Another problem of word vectors is that any word might actually have multiple senses, while vectors are just point estimates. If we wanted to be correct, we needed first to find the right sense for each word in a phrase and only then assign the vector. There is research in "on-the-fly" word vectors that adapt to context, but it's much harder to use. There has been work on representing words not as vectors but as multimodal Gaussian distributions in order to try to deal with polysemy, such as [0], of which an implementation (which I have not tried to use) is available on GitHub at [1]. >It feels different than 3d-space because each point has a shortcut to other points, each point leads to hundreds of other places which might be far apart. It's like a kaleidoscope where a small change can create a very different perspective. I appreciate t