The great convention of deep learning thinkers
2 points · Open discussionsoumith
convnet-benchmarks
Python2014–2025
soumith / convnet-benchmarks
convnet-benchmarks
Easy benchmarking of all publicly accessible implementations of convnets
29Distinct people
36Mentions
0Explicit recommenders
2.7KGitHub stars
What to know before exploring
What it is
Easy benchmarking of all publicly accessible implementations of convnets
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
- autumnai/deep-learning-benchmarks · linked in 2 of the same discussions
Mentioned in the same Hacker News threads. That is not proof they are substitutes.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:03:16 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year36 dated mentions
Observed mentions: 2014-09-29 18:59:16 UTC to 2025-11-07 05:52:19 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 9 | 2 | 7 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 20 | 2 | 18 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 1 | 1 | 0 |
From the project README
An excerpt from the repository, fetched Nov 13, 2026.
Read the project’s introduction
convnet-benchmarks
Easy benchmarking of all public open-source implementations of convnets.
A summary is provided in the section below.
Machine: 6-core Intel Core i7-5930K CPU @ 3.50GHz + NVIDIA Titan X + Ubuntu 14.04 x86_64
Imagenet Winners Benchmarking
I pick some popular imagenet models, and I clock the time for a full forward + backward pass. I average my times over 10 runs. I ignored dropout and softmax layers.
Notation
Input is described as {batch_size}x{num_filters}x{filter_width}x{filter_height}. Where batch_size is the number of images used in a minibatch, num_filters is the number of channels in an image, filter_width is the width of the image, and filter_height is the height of the image.
One small note:
The CuDNN benchmarks are done using Torch bindings. One can also do the same via Caffe bindings or bindings of any other library. This note is here to clarify that Caffe (native) and Torch (native) are the convolution kernels which are present as a default fallback. Some of the frameworks like TensorFlow and Chainer are benchmarked with CuDNN, but it is not explicitly mentioned, and hence one might think that these frameworks as a whole are faster, than for example Caffe, which might not be the case.
AlexNet (One Weird Trick paper) - Input 128x3x224x224
Library
Class
Time (ms)
forward (ms)
backward (ms)
CuDNN[R4]-fp16 (Torch)
cudnn.SpatialConvolution
71
25
46
Nervana-neon-fp16
ConvLayer
78
25
52
CuDNN[R4]-fp32 (Torch)
cudnn.SpatialConvolution
81
27
53
TensorFlow
conv2d
81
26
55
Nervana-neon-fp32
ConvLayer
87
28
58
fbfft (Torch)
fbnn.SpatialConvolution
104
31
72
Chainer
Convolution2D
177
40
136
cudaconvnet2*
ConvLayer
177
42
135
CuDNN[R2] *
cudnn.SpatialConvolution
231
70
161
Caffe (native)
ConvolutionLayer
324
121
203
Torch-7 (native)
SpatialConvolutionMM
342
132
210
CL-nn (Torch)
SpatialConvolutionMM
963
388
574
Caffe-CLGreenTea
ConvolutionLayer
1442
210
1232
Overfeat [fast] - Input 128x3x231x231
Library
Class
Time (ms)
forward (ms)
backward (ms)
Nervana-neon-fp16
ConvLayer
176
58
118
Nervana-neon-fp32
ConvLayer
211
69
141
CuDNN[R4]-fp16 (Torch)
cudnn.SpatialConvolution
242
86
156
CuDNN[R4]-fp32 (Torch)
cudnn.SpatialConvolution
268
94
174
TensorFlow
conv2d
279
90
189
fbfft (Torch)
SpatialConvolutionCuFFT
342
114
227
Chainer
Convolution2D
620
135
484
cudaconvnet2*
ConvLayer
723
176
547
CuDNN[R2] *
cudnn.SpatialConvolution
810
234
576
Caffe
ConvolutionLayer
823
355
468
Torch-7 (native)
SpatialConvolutionMM
878
379
499
CL-nn (Torch)
SpatialConvolutionMM
963
388
574
Caffe-CLGreenTea
ConvolutionLayer
2857
616
2240
OxfordNet [Model-A] - Input 64x3x224x224
Library
Class
Time (ms)
forward (ms)
backward (ms)
Nervana-neon-fp16
ConvLayer
254
82
171
Nervana-neon-fp32
ConvLayer
320
103
217
CuDNN[R4]-fp16 (Torch)
cudnn.SpatialConvolution
471
140
331
CuDNN[R4]-fp32 (Torch)
cudnn.SpatialConvolution
529
162
366
TensorFlow
conv2d
540
158
382
Chainer
Convolution2D
885
251
632
fbfft (Torch)
SpatialConvolutionCuFFT
1092
355
737
cudaconvnet2*
ConvLayer
1229
408
821
CuDNN[R2] *
cudnn.SpatialConvolution
1099
342
757
Caffe
ConvolutionLayer
1068
323
745
Torch-7 (native)
SpatialConvolutionMM
1105
350
755
CL-nn (Torch)
SpatialConvolutionMM
3437
875
2562
Caffe-CLGreenTea
ConvolutionLayer
5620
988
4632
GoogleNet V1 - Input 128x3x224x224
Library
Class
Time (ms)
forward (ms)
backward (ms)
Nervana-neon-fp16
ConvLayer
230
72
157
Nervana-neon-fp32
ConvLayer
270
84
186
TensorFlow
conv2d
445
135
310
CuDNN[R4]-fp16 (Torch)
cudnn.SpatialConvolution
462
112
349
CuDNN[R4]-fp32 (Torch)
cudnn.SpatialConvolution
470
130
340
Chainer
Convolution2D
687
189
497
Caffe
ConvolutionLayer
1935
786
1148
CL-nn (Torch)
SpatialConvolutionMM
7016
3027
3988
Caffe-CLGreenTea
ConvolutionLayer
9462
746
8716
Layer-wise Benchmarking (Last Updated April 2015)
Spatial Convolution layer (3D input 3D output, densely connected)
forward + backprop (wrt input and weights)
Original Library
Class/Function Benchmarked
Time (ms)
forward (ms)
backward (ms)
fbfft
SpatialConvolutionCuFFT
256
101
155
cuda-convnet2 *
ConvLayer
977
201
776
cuda-convnet**
pylearn2.cuda_convnet
1077
312
765
CuDNN R2 *
cudnn.SpatialConvolution
1019
269
750
Theano
CorrMM
1225
407
818
Caffe
ConvolutionLayer
1231
396
835
Torch-7
SpatialConvolutionMM
1265
418
877
DeepCL
ConvolutionLayer
6280
2648
3632
cherry-picking****
best per layer
235
79
155
This table is NOT UPDATED For TITAN-X. These numbers below were on Titan Black and are here only for informational and legacy purposes.
Original Library
Class/Function Benchmarked
Time (ms)
forward (ms)
backward (ms)
Theano (experimental)***
conv2d_fft
1178
304
874
Torch-7
nn.SpatialConvolutionBHWD
1892
581
1311
ccv
ccv_convnet_layer
809+bw
809
Theano (legacy)
conv2d
70774
3833
66941
* indicates that the library was tested with Torch bindings of the specific kernels.
** indicates that the library was tested with Pylearn2 bindings.
*** This is an experimental module which used FFT to calculate convolutions. It uses a lot of memory according to @benanne
**** The last row shows results obtainable when choosing the best-performing library for each layer.
L1 - Input: 128x128 Batch-size 128, Feature maps: 3->96, Kernel Size: 11x11, Stride: 1x1
L2 - Input: 64x64 Batch-size 128, Feature maps: 64->128, Kernel Size: 9x9, Stride: 1x1
L3 - Input: 32x32 Batch-size 128, Feature maps: 128->128, Kernel Size: 9x9, Stride: 1x1
L4 - Input: 16x16 Batch-size 128, Feature maps: 128->128, Kernel Size: 7x7, Stride: 1x1
L5 - Input: 13x13 Batch-size 128, Feature maps: 384->384, Kernel Size: 3x3, Stride: 1x1
The table is ranked according to the total time forward+backward calls for layers (L1 + L2 + L3 + L4 + L5)
Breakdown
forward
Columns L1, L2, L3, L4, L5, Total are times in milliseconds
Original Library
Class/Function Benchmarked
L1
L2
L3
L4
L5
T
Read the full README on GitHubWhat Hacker News says
36 sourcesI mentioned theory and experts, you said enough theory.
I switched to practical applications and walk the talk showing my code, and then you back off and want to go back to opinions.
I see now that you want to back myself with experts since reproducible code and runnable benchmarks is not enough.
Apparently you recognize Nvidia as an expert so let's talk about CuDNN where optimizing convolution is all about memory layout, source: https://github.com/soumith/convnet-benchmarks/issues/93#issu... and it's not about C vs C++ vs PTX.
Or let's hear about what Nvidia says about optimizing GEMM: https://github.com/NVIDIA/cutlass/blob/master/media/docs/eff..., it's all about memory locality and tiling.
Or maybe Stanford, the US government and Nvidia Research are also wrong when pouring significant research in Legion? https://legion.stanford.edu/
>
Read the rest
Legion is a data-centric parallel programming system for writing portable high performance programs targeted at distributed heterogeneous architectures. Legion presents abstractions which allow programmers to describe properties of program data (e.g. independence, locality). By making the Legion programming system aware of the structure of program data, it can automate many of the tedious tasks programmers currently face, including correctly extracting task- and data-level parallelism and moving data around complex memory hierarchies. A novel mapping interface provides explicit programmer controlled placement of data in the memory hierarchy and assignment of tasks to processors in a way that is orthogonal to correctness, thereby enabling easy porting and tuning of Legion applications to new architectures.
Are you saying they should have just called it a day once they were done with C++?
Or you can read the DaCE paper on how to beat CuBLAS and CuDNN: https://arxiv.org/pdf/1902.10345.pdf, it's all about data movement. 6.4 Case Study III: Quantum Transport to optimize transistors heat dissipation, Nvidia strided matrix multiplication was improved upon by over 30%, and this part is pure Assembly, the improvement was about better utilizing the hardware caches.
I'm curious about the NHWC layout they mentioned.
AFAIK CuDNN always had optimizations for NCHW and that was one of Tensorflow speed issue when they choose to default on NHWC, plus the related issues on writing transformation pipelines.
So what does NHWC enables that is new?
Relevant in-depth discussion including CuDNN team lead, Julien Demouth, and Scott Gray who implemented Winograd convolution for Nervana Neon (which interestingly was CHWN so batch last): https://github.com/soumith/convnet-benchmarks/issues/93#issu...
In PyTorch 1.5For those who aren't familiar, Scott Gray (one of the mentors) is a complete genius. Two years after he left Nervana his neural network kernels (which AFAIK haven't been updated since) are still the fastests available: https://github.com/soumith/convnet-benchmarks
In Shannon Labs – $100K Fellowship to support independent researchersI’m doing research (not deployment) and have the same feeling. PyTorch has inspired a blog post [1], Tensorflow didn’t.
Briefly, the benefits of PyTorch are
* easy conversion to NumPy arrays (meaning rest of Python can be used!). This is a bottleneck in Tensorflow; for reasonable sizes, PyTorch is 1000x faster.
* trackbacks are easy to follow (because defines graph by running)
* it’s as fast as tensorflow [2] (or at least torch is, which calls the same C functions as PyTorch, and there’s a tweet [4] by a core dev saying to expect the same speeds). Plus on the web I’ve only found anecdotes that support PyTorch faster than tensorflow.
* it’s easy to extend; everything is a simple Python class. e.g., see their different optimizers [3]
[1]:http://stsievert.com/blog/2017/09/07/pytorch/
[2]:https://github.com/soumith/convnet-benchmarks
[3]:http
Read the rest
There's a saying: lies, damn lies and benchmarks. In this case AFAIK mi one has replicated these claims.
MS hasn't got it in the standard CNN inference benchmark yet though: https://github.com/soumith/convnet-benchmarks. This is the benchmark that showed how slow TensorFlow was on non-Google libraries until Google fixed it.
In Cognitive Toolkit beta for deep learning advancesThere is another nice one here:
https://github.com/soumith/convnet-benchmarks
In Simple Deep Learning BenchmarkWhy can't Torch utilize more threads in CPU cores?
Taken from the article itself:
"both of them cannot run normally when threads usage is set to be bigger than the number of CPU cores on desktop CPU."
Do the authors set up the system correctly?
You're right that Torch is faster than TensorFlow in RNN. But Torch is slower than TesnorFlow in AlexNet and ResNet.
There is a set of benchmarks for many DL approaches as found in https://github.com/soumith/convnet-benchmarks
In Benchmarking State-Of-the-Art Deep Learning Software ToolsNot even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning.
The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf
In comparison, a single Knights Landing Xeon Phi will be ~$7K. I know where I put my money. Caveat Emptor.
But Xeon Phi and I go way back here. They've been trying to beat my AMBER GPU code since 2013 or so. Many man years later I believe that a Knight's Corner is now ~35% faster than 2 Xeon CPUs with 1M atoms or more (source: http://adsabs.harvard.edu/abs/2016CoPhC.201...95N)
Meanwhile, the CUDA code has continued to scale with the GP
Read the rest
U roadmap and a Titan XP is arguably 9-10x faster than 2 Xeon CPUs. No data is supplied at the low-end for Xeon Phi and I think we can safely assume it's because performance there sucks. (source: http://ambermd.org/gpus/benchmarks.htm)
Xeon Phi? IMO avoid avoid avoid until they start winning head to head 3rd party benchmarking fights like Soumith Chintala's fantastic convnet benchmark data: https://github.com/soumith/convnet-benchmarks
No, it's not. Why does TensorFlow have separate benchmarks for Soumith's convnet comparison that show it to be significantly slower?
https://github.com/soumith/convnet-benchmarks
In Google Sprints Ahead in AI Building Blocks, Leaving Rivals WaryWhat's Deep Learning 4J's performance on the Soumith benchmarks?
https://github.com/soumith/convnet-benchmarks
In Generating Recommendations at Amazon Scale with Apache Spark and Amazon DSSTNEThese are the benchmarks supported by Soumith Chintala (Facebook AI Research):
https://github.com/soumith/convnet-benchmarks
They place TensorFlow performance on par with Torch (within 10%).
In TensorFlow – Consise Examples for BeginnersAll of the comparable tools are practically locked into nVidia gpus/CUDA. TensorFlow is rapidly reaching performance parity on that hardware [1] and can now be run on this so it's actually sort of the least locked in framework.
[1] It has been climbing the charts at https://github.com/soumith/convnet-benchmarks for example.
In Google supercharges machine learning tasks with TPU custom chipArria 10: http://www.nextplatform.com/2015/08/27/microsoft-extends-fpg...
600 images/s inference (highest ever publically reported), projected to be as high as 900 images/s. ~$5000 per FPGA.
Compare and contrast with TitanX/GTX 980 TI now topping 5000 images/s. GTX 1080 will only be faster than this and $600.
https://github.com/soumith/convnet-benchmarks
And so far, no FPGA training numbers. But here are some distributed training numbers for $10Kish Xeon servers:
https://software.intel.com/en-us/articles/caffe-training-on-...
64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xe
Read the rest
on servers.
FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.
The most interesting thing about AMD is their ability to create GPUs with superior synthetic specs than their NVIDIA competitor and yet they lose across the board in anything but embarrassingly parallel applications. Not only do they lose, but they lose big.
Here are two examples. First, the engine behind Folding@Home:
https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHF...
See also AlexNet (Deep Learning's most prominent benchmark):
https://github.com/amd/OpenCL-caffe
versus GTX TitanX:
https://github.com/soumith/convnet-benchmarks
TLDR: AMD losing by a factor of 2 or more...
So unless this changes dramatically, NVIDIA IMO will continue to dominate.
But hey, they both crush Xeon Phi and FPGAs so it doesn't suck to be #2 when #3 is Intel.
In NVIDIA Announces the GeForce GTX 1000 SeriesShowing the 15 most recent of 36 sources. Enable JavaScript to search and browse every indexed mention.