dramatiq

> The Many Problems with Celery: — https://steve.dignam.xyz/2023/05/20/many-problems-with-celer... > The problems with (Python’s) Celery: — https://docs.hatchet.run/blog/problems-with-celery > Dramatiq motivation: — https://dramatiq.io/motivation.html Here are some alternatives: Dramatiq: https://github.com/Bogdanp/dramatiq RQ: https://github.com/rq/rq Huey: https://github.com/coleifer/huey Hatchet: https://github.com/hatchet-dev/hatchet

Can't speak for op, but for us the ones based on RabbitMQ have performed well: https://github.com/Bogdanp/dramatiq Stay away from Celery if you can, or stick to v3.2 because 4.x has a ton of bugs.

We tried this at WakaTime and it didn't scale well. We also tried https://github.com/closeio/tasktiger but the only ones that work for us are based on RabbitMQ: Celery<4.0 https://github.com/Bogdanp/dramatiq

Having used pure pip + virtualenv{,wrapper}, pip-tools + virtualenv, poetry and Pipenv for medium to large applications, I'm going to be sticking to pip-tools for the time being for apps. Poetry is fine, but pip-tools is faster and there's less to learn. Pipenv is unbearably slow for large applications and often buggy. For libraries, I've been using Poetry for molten[1] and pure setuptools for dramatiq[2] and, at least for my needs, pure setuptools seems to be the way to go. [1]: https://github.com/Bogdanp/molten [2]: https://github.com/Bogdanp/dramatiq

If you liked Celery 3.x but don't like all the bugs in Celery 4.x, check out these alternative Python task queues: https://github.com/closeio/tasktiger (supports subqueues to prevent users bottlenecking each other) https://github.com/Bogdanp/dramatiq (high message throughput)