benchmarking-public

> Generally not that much has happened in 5 years, sometimes 10-15% improvements are posted that are later offset by bloat. Sorry but unless your workload is some C API numpy number cruncher that just does matmuls on the CPU, that's probably false. In 3.11 alone, CPython sped up by around 25% over 3.10 on pyperformance for x86-64 Ubuntu. https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-fas... 3.14 is 35-45% faster than CPython 3.10 for pyperformance x86-64 Ubuntu https://github.com/faster-cpython/benchmarking-public These speedups have been verified by external projects. For example, a Python MLIR compiler that I follow has found a geometric mean 36% speedup moving from CPython 3.10 to 3.11 (page 49 of https://github.com/EdmundGoodman/masters-project-report) Another academic benchmark here observed an around 1.8x speedup on their

Simon Willison is a great guy, but he's not a Python core developer and his ad hoc benchmark is not what the CPython core team members are using. For the latter, see https://github.com/faster-cpython/benchmarking-public

Why didn't this regression in baseline performance show up (or did it?) on the faster-cpython benchmarks page [0]? Could the benchmarks be improved to prevent similar issues in the future? [0] https://github.com/faster-cpython/benchmarking-public

Python development is done in public so you can just benchmark against the development version to see its improvement. In fact, daily benchmarks are already posted to [1]; it indicates around 20% improvement (corresponding to 1.25x in the table) since 3.10. The only thing you can't easily verify is that whether GIL was indeed historically necessary in the past. [1] https://github.com/faster-cpython/benchmarking-public

Ohhh interesting! So like ML Perf or Faster CPython https://github.com/faster-cpython/benchmarking-public?

Benchmarks aren't too promising[0]. I wonder if the original 500% improvement they targeted at the start of the `faster cpython` project is still a realistic target. [0] https://github.com/faster-cpython/benchmarking-public

Benchmarks aren't too promising[0]. I wonder if the original 500% improvement they targeted at the start of the `faster cpython` project is still a realistic target. [0] https://github.com/faster-cpython/benchmarking-public

It is unclear to me whether Python 3.12 will receive significant improvements. Based on the information from https://github.com/faster-cpython/benchmarking-public, it appears that there may be a 2% performance enhancement. Is this the anticipated result, or are there additional developments awaiting merger? Initially, the "Shannon Plan" (https://github.com/markshannon/faster-cpython/blob/master/pl...) aimed for a 50% improvement with each release. Has this goal been deemed unattainable, or are there adjustments being made to the plan?

How does that reconcile with the benchmarks here, which say that Python 2.12 is currently somewhere between 5% slower to 5% faster? https://github.com/faster-cpython/benchmarking-public