benchmarks

Thank you for cross-referencing that, the data does look accurate and my statement now seems exaggerated. I do think we need skepticism on the A100 charts though until third party benchmarks. > In my experience NVidia benchmark numbers in deep learning are rarely lies - they are highly optimised, in optimal conditions and rarely achievable in the real world. Right, but Nvidia claimed 1360 images/sec for resnet-50 on imagenet. To my knowledge this still hasn’t been realized by a third party. It also isn’t a 4x improvement for fp32 vs fp16, that’s comparing to previous generation. Improvement is more like 1.5x: https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v... Even in very simple synthetic benchmarks the speed up is only 2x: https://github.com/tensorflow/benchmarks/issues/77#issuecomm... I have not seen any benchmarks showing

> Unfortunately the performance charts are completely devoid from reality, and in particular the discussion on tensorcores may be true from an instruction count perspective but does not reflect any third-party benchmark I’ve seen. For example: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... The performance numbers posted here appear to almost exactly reflect the LambdaLabs numbers. Lambda Labs: the RTX 2080 Ti is 96% as fast as Titan V, 73% as fast as Tesla V100 (32 GB) timdettmers: RTX 2080 Ti normalized to 1, Titan V looks about 1.1 to 1.2, V100 is just below 1.5 > for example, see this thread from right after Nvidia announced an 8x improvement in speed on imagenet in tensorflow for V100 Well a non-NVidia person managed to get it up to just above 4x improvement without "using unreleased libraries from NVIDIA". From the s

I appreciate the time and care that went into this post, and there’s a nice discussion of various features. Unfortunately the performance charts are completely devoid from reality, and in particular the discussion on tensorcores may be true from an instruction count perspective but does not reflect any third-party benchmark I’ve seen. For example: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks.... Nvidia has a history of straight-up-lying about tensorcore and other benchmarks (for example, see this thread from right after Nvidia announced an 8x improvement in speed on imagenet in tensorflow for V100: https://github.com/tensorflow/benchmarks/issues/77) In general, fp16 is only 30-40% faster than fp32, and occasionally 2x in really optimal conditions.

Where specified "fp16", the V100 benchmarks use the code from https://github.com/tensorflow/benchmarks/tree/master/scripts... with the flag --use_fp16=true which enables fp16 for some but not all Tensors.