maxas

GPU's don't do any OoO processing like modern CPU's do. They also don't do any register renaming. They execute things really literally, up to the point where one has to manually put delay slots for pipelined stuff if one really writes the raw asm (Which the manufacturers tend to keep really hidden, in order to avoid the binary compatibility trap, see https://github.com/NervanaSystems/maxas as an example for third party assembler for nvidia Maxwell arch) On GPU's the binary compatibility issue is solved by having the driver compile the shader/compute kernel before it's used. As an example nvidia uses PTX (see http://docs.nvidia.com/cuda/parallel-thread-execution/) as an intermediate language in CUDA which is then compiled by the runtime into the actual ASM. On modern CPU's the register renaming has already decoupled the physical registers

"As for "getting my facts straight" I was using actual Titan X numbers I have seen for real applications (e.g. nbody simulation)... it only gets around 4000GFLOPs single precision compared to their advertised 7000." Nonsense. I regularly get ~5.5 TFLOPS out of them running cuBLAS SGEMMs in neural networks. You can get that down to ~1.4 if you do your best to choose stupid small values for m, n, and k, but that's a relatively minor bug and I think it's fixed in cuDNN's latest kernels. But even if it isn't, if you're willing to download Scott Grey's maxas: https://github.com/NervanaSystems/maxas, you can hit 6.4 TFLOPS with his hand-coded SGEMM. Similarly, one can do the same for convolutional layers with Andrew Lavin's maxDNN: https://github.com/eBay/maxDNN Your hubris is amusing, but I reiterate that until you have a shipping chip, all

Looking at some of the details in https://github.com/NervanaSystems/maxas/wiki/Control-Codes reminds me somewhat of the Itanium: a very wide architecture that is capable of high throughput for specialised applications, but requires a lot of software-level support to even work correctly (e.g. consider dependencies.) The fact that it's not well-documented is another similarity. It would be great if nVidia released more documentation, because chances are developers could squeeze even more performance out of their hardware that way.

PTX is a virtual ISA that is not the same as the machine code that runs on the device. In his Introduction[1] wiki page, he says that he studied sgemm implementations and came to the conclusion that NVidia is not using PTX, but an assembler for the real ISA which is not distributed to developers. He claims that his sgemm implementation is almost 5% faster than NVidia's and its faster than anything that can be done in PTX. [1] https://github.com/NervanaSystems/maxas/wiki/Introduction