I've wanted to do that for a while. Look into structured logging. I want/need machines to analyze my logs, and that requires key value pairs. I put any dynamic text in its own field. "error for user 42, remote api timed out" -> level: error, userid: 42, message: api timeout, http_request: <curl call to reproduce>, timestamp: <curr_time>, time_duration_ms: 30000, ... (but as json).
Now I can see what users are affected by which errors however often. Alerts are now trivial to implement. They key is being able to debug an issue after the fact. I like to include a copy-pastable curl call when I can so I can manually reproduce an issue easily as an example