seq2seq

I'm not a Googler/Deepminder, so I can't comment about what goes on inside the company. However, Google is making a hardware investment in Tensor Processing Units. These presumably offer hardware acceleration for the static computation graphs that TensorFlow produces, and PyTorch wouldn't be any good with them. You're right that--as of this time of writing--there are no good seq2seq with attention models in Keras. I think there are a few attempts on Github, but I haven't tried them yet. I don't know anybody else that has tried seq2seq w/ attention in Keras yet either. Additionally, TensorFlow has a seq2seq module, and it does come with an attention mechanism. See https://github.com/google/seq2seq/blob/master/seq2seq/models... Anyway, I think the best thing for folks like you and me is to just keep using PyTorch for research work, and use

They mentioned tensor2tensor, how is this related to another repo: seq2seg:https://github.com/google/seq2seq? Which one is more general?

I see - you're talking about a use case like this: https://github.com/google/seq2seq/blob/4c3582741f846a19195ac... I agree that you have to use a tf.while_loop in those cases. But then tf.scan isn't an option, so I don't understand what you mean by 'quit early' or 'saves time'. When tf.scan is possible, i.e. when you have an input sequence you want to scan over, it is a perfectly good option.

Google has released some framework for translation: https://github.com/google/seq2seq/ DeepMind has also released some framework which also has all the building blocks for translation: https://github.com/deepmind/sonnet

TL;DR - It's the code from "Massive Exploration of Neural Machine Translation Architectures", where they run extensive hyperparameter search using 250K hours of GPU time. Direct GitHub link: https://github.com/google/seq2seq Arxiv: https://arxiv.org/abs/1703.03906