Using LLMs to classify noisy alerts is a really clever approach to tackling alert fatigue! Are you fine tuning your own model to differentiate between actionable and noisy alerts?
I'm also working on an open source incident management platform called Incidental (https://github.com/incidentalhq/incidental), slightly orthogonal to what you're doing, and it's great to see others addressing these on-call challenges.
Our tech stacks are quite similar too - I'm also using Python 3, FastAPI!