models

from their note on github page [1]: > The time required to train the Show and Tell model depends on your specific hardware and computational capacity. > In this guide we assume you will be running training on a single machine with a GPU. In our experience on an NVIDIA Tesla K20m GPU the initial training phase takes 1-2 weeks. The second training phase may take several additional weeks to achieve peak performance (but you can stop this phase early and still get reasonable results). > It is possible to achieve a speed-up by implementing distributed training across a cluster of machines with GPUs, > but that is not covered in this guide. > Whilst it is possible to run this code on a CPU, beware that this may be approximately 10 times slower. So, I assume it will take veeery long time to train it on a MBP, unless they publish their pre-trai

The following appears in the README.md # Run the eval. Try to avoid running on the same matchine as training. Why should one avoid that? https://github.com/tensorflow/models/tree/master/textsum

Agreed, it seems they really hand-picked some shining examples for this post, and it would have been more interesting to see the full spectrum of when it works and when it doesn't. Perhaps the README in the Github repo is a bit more honest in terms of representativeness, though it only has 4 examples, one of them is an interesting failure: article: novell inc. chief executive officer eric schmidt has been named chairman of the internet search-engine company google . human: novell ceo named google chairman machine: novell chief executive named to head internet company https://github.com/tensorflow/models/tree/master/textsum (It's still great that it beats all competition on the ROUGE score, of course.)

You can use Protocol Buffers for configs without having to serialize them in its binary format. Protocol Buffers has always had its own text format [0] and now a JSON mapping [1] as well. The proto text format is actually more flexible and less verbose than JSON since it does not require the outer enclosing set of braces and quotes around all the keys and has support for comments. Here are a couple of examples of config files using the Protocol Buffers text format: * Bazel CROSSTOOL: https://github.com/bazelbuild/bazel/blob/master/tools/cpp/CR... * SyntaxNet: https://github.com/tensorflow/models/blob/master/syntaxnet/s... [0] https://developers.google.com/protocol-buffers/docs/overview... [1] https://developers.google.com/protocol-buffers/docs/proto3#j...

Why not just define your schema using Protocol Buffers and use the [proto3 JSON mapping](https://developers.google.com/protocol-buffers/docs/proto3#j...)? On a similar note, text protos actually work very well as configuration files and are used by SyntaxNet/Parsey McParseface [1] and Bazel's CROSSTOOL config [2]. [1] https://github.com/tensorflow/models/blob/master/syntaxnet/s... [2] https://github.com/bazelbuild/bazel/blob/master/tools/cpp/CR...

What wodenokoto said, and also look up "Word Embeddings", word2vec is a popular method. https://www.tensorflow.org/versions/r0.7/tutorials/word2vec/... There's a bunch of blogs, tutorials, etc, around word2vec and other methods of generating vectors from a training set of words. Also, in the tensorflow models codebase where this syntaxnet code lives, there is an another tensorflow-using-method of generating word embeddings with demonstration code called Swivel https://github.com/tensorflow/models/tree/master/swivel

Parsey McParseface? Nice touch Google. https://github.com/tensorflow/models/tree/master/syntaxnet/s...

Those are the part-of-speech tag accuracies. spaCy's accuracy on the PTB evaluation is 92.2% --- so it makes 20% more errors than P. McP. On the other hand, spaCy is about 200x faster. I've been watching the line of research in SyntaxNet closely, and have been steadily working on replacing spaCy's averaged perceptron model with a neural network model. This is one of the main differences between spaCy and Parser McParseface. The key advantage of the neural network is that it lets you take advantage of training on lots and lots more text, in a semi-supervised way. In a linear model, you grow extra parameters when you do this. The neural network stays the same size --- it just gets better. So, you can benefit from reading the whole web into the neural network. This only works a little bit in the linear model, and it makes the resulting model

For those wondering: the license appears to be Apache 2.0 (https://github.com/tensorflow/models)

It's already available here - https://github.com/tensorflow/models/tree/master/syntaxnet echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh Input: Bob brought the pizza to Alice . Parse: brought VBD ROOT +-- Bob NNP nsubj +-- pizza NN dobj | +-- the DT det +-- to IN prep | +-- Alice NNP pobj +-- . . punct

It's also reasonably similar to tf.slim, which we haven't released as a standalone supported component yet, but that's included in the release of the trainable inception model -- https://github.com/tensorflow/models/tree/master/inception/i... I think it's great that we're seeing a lot of experimentation in how to express models for machine learning. It's very clear we haven't found the best ways yet, and seeing what people choose to try to make easy to express (and harder) is good for progress. (I'm still grouchy about the state of modularity within machine learning models. It's not easy enough to reuse / have libraries at the model level yet.)