so i mean what e.g. conan does is acknowledge this and require that package creators provide "recipes" that tell conan how to build the project.
buckaroo seems to take a similar approach: https://github.com/LoopPerfect/buckaroo/wiki/Creating-a-Pack...
i will say that in my time using conan to attempt to solve for compiling an application for x86 and a few flavors of ARM, this is NOT a silver bullet.. a huge fraction of third party libraries we tried to pull in needed to have their recipes modified to build correctly.