I have to admit the solution you describe was a pleasure to read because that is exactly what I'm building :) promotion was not my goal here but would happily share a link to the beta if you're interested.
edit: will add the link as well since your description is so on spot (other than my app uses the notifications action menu for the tracking). It's for ios and still in beta but pretty stable so feel free to sign up. would love to hear if you found it (at least potentially) useful https://www.t12n.io