Far less scalable, but it is dramatically simpler to deploy. Often gets you surprisingly far though. Would be interesting to know how many monitored integrations could be supported by that flow.
Not to mention that having multiple components doesn't mean it's "scalable" by default, it could happen that some part of the pipeline doesn't like multiple instances of something.
Doesn't have all the plumbing you'd want, there is a wrapper (https://github.com/bretth/django-pq/) that seems to give you an entrypoint command more like `celery worker ...` but I've not investigated it closely.
A simple way to use an RDBMS as a message queue, that has been in use since before most HN readers were born, is roughly:
- enqueue an item by inserting a row into a table with a status of QUEUED
- use a SELECT FOR UPDATE, or UPDATE...LIMIT 1, or similar, to atomically claim and return the first status=QUEUED item, while setting its status to RUNNING (setting a timestamp is also recommended)
- when the work is complete, update the status to DONE
The first software company I worked for was using this basic approach to queue outbound emails (and phone and fax... it was 2005!), millions per day, on an Oracle DB that also ran the entire rest of the business. It's not hard.