WeightWatcher

Here's an alternative approach, that actually provides real world results https://calculatedcontent.com/2019/12/03/towards-a-new-theor... Using techniques from statistical mechanics and strongly correlated systems, we can compute the average-case-behavior of a real world DNN We believe we can reproduce some of the results of the NTK by using a Gaussian Random Matrix. But if we use a more realistic, heavy tailed matrix, we get more practical results A early fork of the theory has been published in JMLR https://arxiv.org/abs/1810.01075 and the empirical results in Nature Communications https://www.nature.com/articles/s41467-021-24025-8 and we have an open source tool , weightwatcher, which can be used in production pip install weightwatcher https://github.com/CalculatedContent/WeightWatcher Please give it a try and let me know if it is us