TCN

This AI toolkit works on popular Intel CPUs, and is a big step forward for the new Intel Nervana Neural Network Processor (NNP-I) hardware chip akin to a GPU. The Intel AI Lab has an introduction to NLP (https://ai.intel.com/deep-learning-foundations-to-enable-nat...) and optimized Tensorflow (https://ai.intel.com/tensorflow/) One surprising research result for this NLP is that a simple convolutional architecture outperforms canonical recurrent networks, often. See: CMU lab, Sequence Modeling Benchmarks and Temporal Convolutional Networks (TCN) https://github.com/locuslab/TCN If you're interested in Nervana, here are some specifics: the chip is for hardware neural network acceleration, for inference-based workloads. Notable features include fixed-point math, Ice Lake cores, 10-nanometer fabs, on-chip memory management by software directly

I haven’t done any significant projects using neural networks for sequence modeling or analysis, but when I do, I plan to start with a temporal convolutional network, based on [0]. They argue that RNNs being standard is likely an artefact of the history of the field, while they get superior performance from TCNs. [0] https://github.com/locuslab/TCN