loop do
DB.transaction do
# pull jobs in large batches
job_batch = StagedJobs.order('id').limit(1000)
if job_batch.count > 0
# insert each one into the real job queue
job_batch.each do |job|
Sidekiq.enqueue(job.job_name, *job.job_args)
end
# and in the same transaction remove these records
StagedJobs.where('id <= ?', job_batch.last).delete
end
end
end
Isn't this essentially a busy loop? You can achieve something much more performant by using `LISTEN` and `NOTIFY` to fire an event every time a row is inserted.Then the enqueuer can do a preliminary scan of the table when it boots up and then just a `LISTEN` instead of polling the DB.