2) Investing in logging early, so that your system is giving you metrics as it grows
3) Understanding that logging and metrics are a conversation that happens with a system over time. You don't do it once and then are done, it's like teaching a child to talk. You have to leave room to expand, and be actively building a 'vocabulary' in your logging and metrics that makes your day-to-day simpler.
4) KISS -- at least on the "big stuff". Your high level architecture should be 'clean' in the sense that issues can be isolated and readily assigned to a subcomponent. I find that isolated communication points that let you bifurcate responsibility are key to quick deductions. Ie the backend publishes data to a store, front-end reads data from that store, so if the data store agrees/disagrees with either you know exactly where the problem is.