paper

other specified index). Python offers access to hard links (on systems that support them) in the standard library. All the filesystem-related stuff is already implemented in C under the hood, and a lot of the remaining slowness of I/O is due to unavoidable system calls. Another big deal is that when uv is asked to precompile .pyc files for the installation, it uses multiple cores. The standard library also has support for this (and, of course, all of the creation of .pyc files in CPython is done at the C level); it's somewhat naive, but can still get most of the benefit. Plus, for the most part the precompiled files are also eligible for caching, and last time I checked even uv didn't do that. (I would not be at all surprised to hear that it does now!) > It totally depends on the problem that you're trying to solve. My point was more that even when you have a reasonable problem, you have to be careful about how you interface to the compiled code. It's better to avoid "crossing the boundary" any more than absolutely necessary, which often means designing an API explicitly around batch requests. And even then your users will mess it up. See: explicit iteration over Numpy/Pandas data in a Python loop, iterative `putpixel` with PIL, any number of bad ways to use OpenGL bindings....

wnloaded an average of 12 million times per day (https://pypistats.org/packages/pip). They're also worth making exactly because Pip still has a very long way to go in terms of performance improvement, and because experiments like this show that the problem is very much with Pip rather than with Python. Tons of people hyping Uv assume that it must be "rocket emoji, blazing fast, sparkle emoji" because it's written in Rust. Its performance is not in question; but the lion's share of the improvement, in my analysis, is due to other factors. Documenting past performance gains helps inform the search for future improvements. They aren't going to start over (although I am: https://github.com/zahlman/paper) so changes need to be incremental, and constantly incorporated into the existing terrible design. Showing off unexpected big-O issues is also enlightening. FTA: > This was the code to sort installed packages just before the final print. > There was a quadratic performance bug lurking in that code. The function `env.get_distribution(item)` to fetch the package version that was just installed was not constant time, it looped over all installed packages to find the requested package. The user would not expect an installation of hundreds of packages to spend a significant amount of time in preparing to state which packages were installed. But Pip has been around since 2008 (https://pypi.org/project/pip/#history) and Ian Bicking may never have imagined environments with hundreds of installed packages, never mind installing hundreds at a time. Finally, documentation like this helps highlight things that have improved in the Python packaging ecosystem, even outside of Pip. In particular: > Investigation revealed the download is done during the dependency resolution. pip can only discover dependencies after it has downloaded a package, then it can download more packages and discover more dependencies, and repeat. The download and the dependency resolution are fully intertwined. This is mostly no longer true. While of course the dependency metadata must be downloaded and cannot appear by magic, it is now available separately from the package artifact in a large fraction of cases. Specifically, there is a standard for package indices to provide that information separately (https://peps.python.org/pep-0658/), and per my discussion with Pip maintainers, PyPI does so for wheels. (Source distributions — called sdists — are still permitted to omit PKG-INFO, and dependency specifications in an sdist can still be dynamic since the system for conditional platform-dependent dependencies is apparently not adequate for everyone. But in principle, some projects could have that metadata supplied for their sdists, and nowadays it's relatively uncommon to be forced to install from source anyway.)

thon explicitly, and in the exceptional cases where the code is depending on environment variables being set (e.g. because it does something like `subprocess.call(['python', 'foo.py'])` to run more code in a new process, instead of checking `sys.executable` like it's supposed to, or because it explicitly checks `VIRTUAL_ENV` because it has a reason to care about activation) then you can set those environment variables yourself. Creating a virtual environment is actually very fast. The built-in `venv` standard library module actually does it faster in my testing than the equivalent `uv` command. The slow part is bootstrapping Pip from its own wheel - but you don't need to do this [2]. You just have to tell `venv` not to, using `--without-pip`, and then you can use a separate Pip (for recent versions — almost the last 3 years now) copy cross-environment using `--python` (it's a hack, but it works if you don't have to maintain EOL versions of anything). If you need heavy-duty support, there's also the third-party `virtualenv` [3]. Much of the same tooling that manages virtual environments for you — in particular, pipx and uv, and in the hopefully near future, PAPER [4] — also does one-off script runs in a temporary virtual environment, installing dependencies described in the script itself following a new ecosystem standard [5]. Uv's caching system (and of course I am following suit) makes it very fast to re-create virtual environments with common dependencies: it has caches of unpacked wheel contents, so almost all of the work is just hard-linking file trees into the new environment. [0]: https://stackoverflow.com/questions/76105218 [1]: https://chriswarrick.com/blog/2018/09/04/python-virtual-envi... [2]: https://zahlman.github.io/posts/2025/01/07/python-packaging-... [3]: https://virtualenv.pypa.io/ [4]: https://github.com/zahlman/paper [5]: https://peps.python.org/pep-0723

be even slower than with pip, since it's using pip behind the scenes and adding its own steps). On the other hand, to my understanding, to use uv as a developer you'll still need to choose and install a build backend such as Flit or Hatchling, or else rely on the default Setuptools. One major reason developers are switching to uv is lockfile support. It's worth noting here that an interoperable standard for lockfiles was recently approved (https://peps.python.org/pep-0751/), uv will be moving towards it, and other tools like pip are moving towards supporting it (the current pip can write such lockfiles, and installing from them is on the roadmap: https://github.com/pypa/pip/issues/13334). If you, like me, prefer to follow the UNIX philosophy, a complete developer toolchain in 2025 looks like: * Python itself (if you want standalone binaries like the ones uv uses, you can get them directly; you can also build from source like I do; if you want to manage Python installations then https://github.com/pyenv/pyenv is solid, or you can use the multi-language https://asdf-vm.com/guide/introduction.html with https://github.com/asdf-community/asdf-python I guess) * Ability to create virtual environments (the standard library takes care of this; some niche uses are helped out by https://virtualenv.pypa.io/) * Package installer (Pip can handle this) and manager (if you really want something to "manage" packages by installing into an environment and simultaneously updating your pyproject.toml, or things like that; but just fixing the existing environment is completely viable, and installers already resolve dependencies for whatever it is they're currently installing) * Build frontend (the standard is https://build.pypa.io/en/stable/; for programmatic use, you can work with https://pyproject-hooks.readthedocs.io/en/latest/ directly) * Build backend (many options here - by design! but installers will assume Setuptools by default, since the standard requires them to, for backwards compatibility reasons) * Support for uploading packages to PyPI (the standard is https://twine.readthedocs.io/en/stable/) * Optional: typecheckers, linters, an IDE etc. A user on the other hand only needs * Some version of Python (the one provided with a typical Linux distribution will generally work just fine; Windows users should usually just install the current version, with the official installer, unless they know something they want to install isn't compatible) * Ability to create virtual environments and also install packages into them (https://pipx.pypa.io/stable/ takes care of both of these, as long as the package is an "application" with a defined entry point; I'm making https://github.com/zahlman/paper which will lift that restriction, for people who want to `import` code but not necessarily publish their own project) * Ability to actually run the installed code (pipx handles this by symlinking from a standard application path to a wrapper script inside the virtual environment; the wrappers specify the absolute path to the virtual environment's Python, which is generally all that's needed to "use" that virtual environment for the program. It also provides a wrapper to run Pip within a specific environment that it created. PAPER will offer something a bit more sophisticated here, for both aspects.)

hniques; since 22.3 you can just use the `--python` flag, but this limits you to environments where the current Pip can run, and re-launches a new Pip process taking perhaps an additional 200ms - but this is still much better than bootstrapping another copy of Pip!) * using heuristics when solving for dependencies (Pip's backtracking resolver is exhaustive, and proceeds quite stubbornly in order) * having a smarter caching strategy (it stores uncompressed wheels in its cache and does most of the "installation" by hard-linking these into the new environment; Pip goes through a proxy that uses some opaque cache files to simulate re-doing the download, then unpacks the wheel again) * not speculatively pre-loading a bunch of its own code that's unlikely to execute (Pip has large complex dependencies, like https://pypi.org/project/rich/, which it vendors without tree-shaking and ultimately imports almost all of, despite using only a tiny portion) * having faster default behaviours; e.g. uv defaults to not pre-compiling installed packages to .pyc files (since Python will do this on the first import anyway) while Pip defaults to doing so * not (necessarily) being weighed down by support for legacy behaviours (packaging worked radically differently when Pip first became publicly available) * just generally being better architected None of these changes require a change in programming language. (For example, if you use Python to make a hard link, you just use the standard library, which will then use code written in C to make a system call that was most likely also written in C.) Which is why I'm making https://github.com/zahlman/paper .

ribed it a bit in https://news.ycombinator.com/item?id=43825508 . I've also written a few posts on my blog (https://zahlman.github.io) about design problems with pip, and going forward I'll be continuing that, along with giving a basic overview of the PAPER design and explaining how it addresses the problems I've identified.

ne script metadata described by https://peps.python.org/pep-0723/, the upcoming lock file standard in https://peps.python.org/pep-0751/, etc.) you become able to reproduce a minimal environment, automate that reproduction, and create an installable sharable package for the code (a "wheel", generally) which you can publish on PyPI - allowing others to install the code into an environment which is automatically updated to have the needed dependencies. Of course, none of this is new with `uv`, nor depends on it. The installer and venv management tool I'm developing (https://github.com/zahlman/paper) is intended to address use cases like yours more directly. It isn't a workflow tool, but it's intended to make it easier to set up new venvs, install packages into venvs (and say which venv to install it into) and then you can just activate the venv you want normally. (I'm thinking of having it maintain a mapping of symbolic names for the venvs it creates, and a command to look them up - so you could do things like "source `paper env-path foo`/bin/activate", or maybe put a thin wrapper around that. But I want to try very hard to avoid creating the impression of implementing any kind of integrated development tool - it's an integrated user tool, for setting up applications and libraries.)