core-to-core-latency

rself: [0] https://github.com/nviennot/core-to-core-latency

ings like scheduling things that talk on "close" cores, and at the micro level by doing things like implementing NUMA aware locking primitives: https://dl.acm.org/doi/10.1145/3477132.3483557 There's also definitely been CPUs (not sure if this is still a thing) where some cores share memory channels and some cores don't so you can access RAM faster (higher bandwidth) if you spread your access between the two sets of cores instead of staying within one.

shit.) (if you have a N100/N300 and could spare a moment, you can answer this question with this tool!) https://github.com/nviennot/core-to-core-latency Similarly, Apple's e-cores are really intended for background workers in phones, not interactive tasks, and not batch processing an intensive workload. AMD's e-cores are literally just slightly lower-clocking/lower-cache/4-core-CCX versions of the normal cores, so, there shouldn't be this cliff. Being inside an e-core CCX should be the same as being on a non-big.little processor entirely, other than going back to a multi-CCX model and having a bit lower performance. Similarly, in the long term, Intel is supposedly looking at something they call "royal cores"/"rentable units" where it's actually (eg) 3x little/1x big core in the same cluster, or some of the units can be paired together to work faster as a single execution stream. So you don't have the latency of flinging a work item all the way to some other stop on the ringbus, because the e-cores are right there and work in the same cache hierarchy/etc. There is some iceberg-work there in terms of making the scheduling play nicely. But oh look, they've been working towards exactly that with the Intel Thread Director (tm) stuff. Everyone thought it was hideously overwrought for just some basic big.little design... and it is. But when you are scheduling 4 threads onto a core and 1 of them goes fast, the CPU itself will need to make the determination. This will actually vary over time too - sometimes you will have threads on the e-cores while you wait for memory access, etc, but then when they're ready they will burst on the p-cores and churn through their work, and go back to the e-cores. (in a way you can argue this is the same vision that xeon phi was built for... xeon phi looks like a processor built for a world of intensive AVX-512 work segments, stitched together with highly-threadable low-performance IO segments/other work. Knight's Landing basically Silverton (e-cores) with SMT4 added (to an e-core, yes) and you get 72 of them, and the bulk of your work is happening in the AVX-512 parts stitched together by these scalar SMT4 threads. Well, Royal Cores are the same idea but with regular scalar code - put the hard parts in the p-cores and the easy parts in the e-cores, and run enough work to saturate it.) But everyone is fishing at this same "p-cores churn through the hot thread and emit bulk work items, e-cores handle the bulk work out of a queue or something" model and that's just not tenable when the e-cores have this massive latency hit/etc. It's not that "e-cores are shit for games" it's that "having half of your processor have twice the latency and exist in a separate cache partition is shit for games"... same reason multi-CCD AMD chips don't do well in games.

c6a.metal. https://github.com/nviennot/core-to-core-latency#amd-epyc-7r... Intel Ice Lake (c6i.metal), being monolithic, starts way worse. Any communication has to traverse a shared ring bus and thus takes 40 minimum. But only peaks at 65ns, I much better than Milan. I https://github.com/nviennot/core-to-core-latency#intel-xeon-... M1 Pro is neat. An 8c chip has three "CCX" alike complexes, 2 perf of 3c each and 1 efficiency of 2c. Smart. Latency is 40ns across cluster, 150ns outside cluster. https://github.com/nviennot/core-to-core-latency#apple-m1-pr... Doing anything off the first socket on AMD is terrible. 90-110ns across CCX on the same CCD, but any communication involving the 2nd CCD at all (even across the CCD itself) is a staggering 190ns to 210ns. https://github.com/nviennot/core-to-core-latency#dual-amd-ep... That's around what the pyramid shows as the upper end for CXL memory (170-250ns)! Please also kindly note these figures probably don't scale linearly with core clockspeed but they probably do scale somewhat; direct comparison is inadvised. But it's good interesting directional data showing some very contemporary latency situations deep in the heart of computing that CXL is unlikely to be better than. Using core to core is a weird proxy but illuminative of how complex & odd it is providing system connectivity is to the smaller CCX core clusters. More on point is talking about main memory latency. Anandtech has great coverage of core to core, and also crucially main memory latency too. There's a lot of nuance & config variance here (NPS0-4) but there's generally a regime where a cluster can be getting around 12ns access (oh that's L3 cache), but it will quickly ramp up to 110-140ns if trying to access wide ranges of memory. It starts to look like a core to core grade speed hit. (Anandtech doesn't cover cross socket memory but it won't be good!) https://www.anandtech.com/show/16529/amd-epyc-milan-review/4 Notably the IOD is basically a Northbridge controller connecting all the individual CCX clusters: key for the talking to other clusters, key to the to talking to memory, key to the exposing PCIe/CXL. If core to core is 150ns say, it's well possible CXL's additional overhead could actually be quite marginal! That's what the article seems to be arguing. Maybe, or not, maybe much penalty will be on top of this hit; too early to tell probably. Damn computing architectures are neat. CXL will add so much new ability to try new things. My main sadness though is just that Nvidia now has 900GBps switched fabric that vastly out stages CXL, which still hasn't shipped 3.0, which is the first switched CXL (aka multiple hosts can share the CXL network); a hypothetical PCIe 6.0 x16 based CXL implementation is comparatively only 121GBps, doesn't exist yet, and isn't switched yet. Then again, AMD EPYC sockets are 128 lane (in 1p mode), so that actually does aggregate to almost 2TBps connectivity... I ran numbers recently & it was not this sunny: I think I'd miscalculated somehow! Very excited for new possibilities. An ecosystem that people can build parts for & try things on, at these great capacities, being near to cores, will be compelling. I hope RISC-V can come play too, ideally with some open source designs (albeit right now everyone is mostly vendored up for their PCIe but CXL hypothetically is simpler).