If the tasks in the queue are not important, then it's fine if they are dropped.
If the tasks in the queue are really important (to the point where dropping one will require remediation) then you would already have set up the necessary tools for introspection and control, even for a few dozen tasks. In particular:
- Detecting that a task has been lost, so it can be re-inserted later.
- Detecting that a task has stalled, so it can be removed and investigated.
- Reordering the queue based on the relative priority of the tasks.
In any case, I would argue that if an upstream system needs the ability to hold some tasks back when the queue is too full, then the queue is in a bad middle ground where it's not completely solving the problem (since "holding back tasks" means a separate queue-like feature in the upstream system) but still consuming a chunk of the infrastructure simplicity budget.
From there, one should either eliminate the queue (replacing it with the "holding back tasks" feature in the upstream system, combined with backpressure from the downstream system) or improve it until it can accept everything the upstream system throws at it.