I'm not sure how applicable it is for books, because generally prices are more uniform for books than they are for software, but in the case of software I wonder if the ones who lose sales to piracy are the makers of more affordable software?
Imagine a world where piracy was effectively impossible for most people, and you want to do some occasional image editing. You look at Photoshop and it is way out of your budget. But you don't actually need most features of Photoshop. You find that Pixelmator Pro can do everything you want, and at $40 you can afford it and it is a good value. So you buy Pixelmator Pro.
In a world where piracy only takes a little time and effort you pirate Photoshop.
In both scenarios Adobe doesn't get any of your money so we can't say that pirating Photoshop in the second scenario cost Adobe a sale. But it did cost Pixelmator Team a sale.
In general, I suspect that the main effect of piracy is to change our allocation of spending within certain categories or groups of categories, without changing the total we spend in those categories.