pyflow

of different problems with running python cli programs. It's an old flag, but too few people know about it. E.G: instead of running "black" or "pylint", do "python -m black" or "python -m pylint". Or course you may want to chose a specific version of python, so "python3.8 -m black" for unix, or "py -3.8 -m black" on windows. To test out the __pypackages__ concept, give a try to the pdm project: https://github.com/pdm-project/pdm At last, some other tools that I wish people knew more about that solves packaging issues: • pyflow (https://github.com/David-OConnor/pyflow): it's a package manager like poetry, but it also installs whatever python you want like pyenv. Except it provides the binary, no need to compile anything. It's a young project with plenty of bugs, but I wish it succeeds because it's really a great concept. Give it a try, it needs users and we would all benefit from it becoming popular as it's a very sane way of setuping a python dev env. • shiv (https://shiv.readthedocs.io/): it leverages the concept of zipapp, see PEP 441 from 2013, meaning the ability that python has to execute code inside a zip file. It's a successor to pex. Basically it lets you bundle your code + all deps inside a zip, like a Java .war file. You can then run the resulting zip, a .pyz file, like if it were a regular .py file. It will unzip on the first execution automatically and run transparently. It makes deployment almost as easy as with golang. • nuitka (https://nuitka.net/): takes your code and all dependencies, turns them into C, and compiles it. Although it does require a bit of setup, since it needs headers and a compiler, it results reliably in a standalone compiled executable that will run on the same architecture with no need for anything else. Also it will speed up your Python program, up to 4 times. In my experience, it's also easier and more robust than pyinstaller, cx_freeze and so on. • pyodide (https://pyodide.org/en/stable/): python compiled to WASM to run in the Web browser. Useless for web programming given the huge size of the runtime, but great for teaching, as it allows students to basically access a zero install full featured python dev env by clicking a link. Try it out, it's awesome, you can even create and query a sqlite db, thanks to the virtual FS: https://notebook.basthon.fr/

rong--whatever Python build tool I use, I'll eventually need support and there's a lot to be said for having a thriving community that has almost always run into my exact problem before. > Python in 3.5 added complete zip app support, which has improved this dramatically from my perspective. Extended by things like shiv https://github.com/linkedin/shiv make it fairly complete. We're currently using this via pex. It mostly works, but we still run into problems occassionally (system dependencies, for example). Figuring out how to integrate these tools into the broader build process is another problem to solve--we're using `pants` which supports pex out of the box, but we're running into lots of bugs or other problems. I'll keep an eye on shiv. > This is interesting to me. I prefer async/await in general, because it has become a standard across programming languages and I find it really easy to reason about. I also find channels to be too widely seen as a cure-all, when the only study so far has shown they actually led to an increase bug count. But I don't discount the value of real-parallelism, and am glad to see that Python has been pushing harder on that lately, with things like subshells that allow bypassing the GIL on a single thread. My biggest issues with async/await are (1) every package needs an async variant (async boto, async docker, etc etc). We work around this by running them in a thread pool executor, and I think that works, but I don't know if I'm holding the GIL unnecessarily and causing performance issues (fundamentally difficult to diagnose). This is roughly the "what color is my function" problem. (2) it's really easy to starve the event loop by calling into something that transitively makes a sync call or otherwise just does a lot of CPU-heavy work. We've run into both kinds in production and they've been really hard to troubleshoot (because the requests that time out often aren't the ones that are actually causing the problems). (3) dynamic typing means it's super easy to forget to await things. Tests should catch this, but we find ourselves writing tests _just_ to catch this (e.g., we now write tests for entrypoints that _just_ `await lib_function(params)`; we would normally not write tests for such simple functions, but now we have to). Static typing is the right way to solve this and mypy does, but mypy has too many other issues (at the moment) for our org. One substantial criticism of goroutines is that they're less safe than async/await because you need to make sure the code you're running is threadsafe. I appreciate this criticism, but I think it's the right tradeoff for Go's performance aspirations (another great high-performance alternative is Rust's borrow checker, but that's the wrong tradeoff for Go's developer productivity aspirations).

anguages and I find it really easy to reason about. I also find channels to be too widely seen as a cure-all, when the only study so far has shown they actually led to an increase bug count. But I don't discount the value of real-parallelism, and am glad to see that Python has been pushing harder on that lately, with things like subshells that allow bypassing the GIL on a single thread.