Unfortunately Wheels don't solve this problem as it seem inherent to how python imports modules.
I've seen people get around this by vendoring their dependencies and I've done some tricks in the past where you provide a vendor proxy package that you can point at a different site packages but this is brittle if your dependency has additional dependencies or uses absolute imports. Now you're maintaining a fork.
I'd love to hear if anyone has had more success in this realm. I love Python but using node and rust, and having the ability to have isolated dependencies trees has made me sad about the state of python packages.