As a manager myself, you have terrible managers. They should be able to acknowledge the facts of the situation and not put the blame on you. To the opposite, good work, such as taking over something critical when someone left the team should have been rewarded. Metrics are useful, but to only rely on them is what lazy or incompetent managers do.