Ah, right, that makes sense. I don't think it's a fair criticism though, as you're crossing the boundaries between packages provided by your language ecosystem vs. those provided by your OS (distribution). There's no easy way to specify this in a way that would work on all platforms or you'd have to start adding logic to pip that knows how to apt/yum/pacman/port install those things.
It's the same if you look at Ruby, Node, even Go and many others. The second you need native libraries/binaries all you have to go on is the hope that the author has documented them along with the rest of the installation instructions. The current only way around it is hoping that your distribution has packaged up the library you need (and is up to date enough for your requirements) in their native format and declaring the required dependencies.