I guess alerts to messengers are OK as long it's only a couple manually created ones, and there should be a graphical dashboard to learn the rest of problems
I guess alerts to messengers are OK as long it's only a couple manually created ones, and there should be a graphical dashboard to learn the rest of problems
We had a rule in my team (before a management change that blew it all to shit) that we don’t use email or messaging for monitoring. Everything goes into the SOR. Once it’s in the SOR, if people want emails, texts, or whatever, it let them know there is work to do, that’s up to the team. Others would make dashboards… lots of options once it’s in the system, and nothing gets lost.
For example, I went from a team that looked at tickets all day to one that mainly worked on user stories in Jira. Because no one was looking at the incidents in the SOR, things were getting missed. I wrote something to check for incident tickets assigned to our team every hour, and it would post them in our team chat so people knew there was work to do. Then once per day, it would post everything still unassigned, so if something was lost on that hourly post, it would annoy everyone once per day until it was assigned/resolved. It worked out decently well. If there was a lot of stuff, it would post a message to have someone actually login to the SOR and look at all our tickets. I would sometimes use the standup to assign stuff out and get some attention on it, if things were getting bad.
But, if alarms are configured in a clean way, ideally your team is getting some warnings and such there and then if there's an alert that needs to actually page, it sends that to PagerDuty or whatever platform you use along with another message to Slack.
What strategies have you seen work well?
I have no idea what's the best setup for small companies