Whilst perhaps a bit tangential to what you're looking for, "Database Reliability Engineering" by Charity Majors is an excellent read, and has a chapter covering all the major steps in improving observability in existing apps / infra (here applied to databases).