I also think expanded notions of violence are appropriate, though excluded from Pinker's analysis. For example, if a rich country COULD provide equal, high-quality health care to all its citizens, but doesn't because the health care industry (hospitals, insurers, physician groups, pharma) lobbies against such outcomes, and excess morbidity and mortality result, that is a kind of violence, though one that Pinker is not interested in.
Or North Korea (or to a lesser extent China), which maintains relatively low levels of violence, but is simultaneously committing abuses against its population.
The idea that fewer people (or a smaller proportion of people) are smashing their neighbor's head in with a rock doesn't necessarily constitute absolute "progress" to me (though it would be a nice development... provided it was true... which it might not be).
But it feels like we're expecting a lot from Pinker. Basically a full analysis of all types of coercion that can possibly occur. That seems a basically impossible task to undertake. Drawing the line somewhere does not seem unfair.
If his conclusion had been, maybe we're seeing a drop in a particular form of violence and we're not entirely sure why but here are my thoughts (ahem, development of the atom bomb perhaps?), I would have agreed with him, but he wouldn't have sold all those books...
Not the OP, but I enjoyed it, and think people should read or listen to it even though it contains some inaccuracies.