LASER

I'm curious how people are handling multi-lingual embeddings. I've found LASER[1] which originally had the idea to embed all languages in the same vector space, though it's a bit harder to use than models available through SentenceTransformers. LASER2 stuck with this approach, but LASER3 switched to language-specific models. However, I haven't found benchmarks for these models, and they were released about 2 years ago. Another alternative would be to translate everything before embedding, which would introduce some amount of error, though maybe it wouldn't be significant. 1. https://github.com/facebookresearch/LASER

Yes but they're not maxpooling in the last dimension. They're max pooling over the sequence length [0], (the other way doesn't really make sense in this context). The output size is 1024, the hidden vector size is 512 but they're using bidirectional LSTMs which concatenates the outputs of each direction -- so the total is 1024 [1]. [0] https://github.com/facebookresearch/LASER/blob/fec5c7d63daa2... [1] https://pytorch.org/docs/stable/nn.html#lstm

Max-Pooling a 1024-cell output LSTM will result in a 1024-sized vector. Looking at the code for the Encoder (https://github.com/facebookresearch/LASER/blob/fec5c7d63daa2...), each LSTM has the same amount of hidden cells. (although the default parameters of that class don't quite match the ones used in the post; so I assume it's 512x2x5).