Adventures in AWS Lambda Land
spacelift.io
spacelift.io
The visibility timeout can be configured anywhere between 0 seconds and 12 hours. While some trial and error may be needed to get your application just right, it's hardly a flaw of SQS.
In my opinion, the fact that the SQS connector can't keep hold of that message after receiving a 429 response (while keeping the visibility state updated, that is something we were doing ourselves prior to this migration), and process it right away after capacity has been freed up, is a flaw.
There's a 5 minute waiting time for the retry in this scenario that needn't be there, even if the lack of it would force the SQS connector to be slightly (but just slightly) more stateful.
If you interact with the SQS API from within your code (whether that's in your Lambda function or some other orchestration like a Step Function), the default handling ceases and you're in full control. Do this and you can now decide exactly and programmatically when a message is made visible again. While it's not the focus of his talk, there's a good example of SQS message visibility done properly here: https://www.youtube.com/watch?v=I85MPC_5yXE&t=2614s
Author here, happy to answer any questions.
TLDR: We moved to Lambda for our async processing, there’s been a few surprises, but it’s been a very good experience overall. The article contains various lessons learned related to local development, secret handling, container images, goroutines in Lambdas, the SQS connector, etc.