Maintaining compatibility was never an option. But the most sensible solution to running out of address space is simply to extend the address space. Instead IPv6 decides to change the format into something which is not really human readable, and decides to kill NAT as well. Which was really a terrible decision imo. I like NAT, I like having addresses that I can remember, and I hate the idea of having a unique globally routable address for every device.
The primary argument for adopting IPv6 is that IPv4 will be exhausted. Not that there’s something good about IPv6 that I would want to have. Personally I hope it never succeeds in getting sufficient adoption, so that eventually we can have a good IPv7 that’s just a bigger version of IPv4.