kann

There have been several posts on tiny/lightweight neural network libraries. They are indeed small, but they only implement the most basic multi-layer perceptrons (MLPs) with little optimization (e.g. they often implement backprop inefficiently). KANN is far beyond MLPs. It implements automatic differentiation on top of generic computational graphs [1], supporting dropout, proper mini-batching, reshape/slice/concat of general n-d arrays, arbitrary weight sharing, 1D/2D convolution/pooling, SGD/RMSprop, RNN/LSTM and graph/weight saving/loading. KANN is like a CPU-only mini-tensorflow. KANN optimizes convolution [2] and matrix multiplication [3] (also optionally calls BLAS' sgemm). It can use SSE and multi-threading when available. On multiple threads, KANN is sometimes faster than CPU-only Theano+Keras. I implemented KANN when I was studyin

kann [0] is a similarly small neural network library written in pure C. It's quite fast for CPU-only, but only applicable for rather small problems. [0] https://github.com/attractivechaos/kann