It also can't be backwards compatible with IPv4 networking and software. The network gear will drop extra address, the OS will ignore it, and software will blow up.
It would be much better to make a new version. But if going to make new protocol, might as well make the address big enough to not need expansion again.
Then you have to update every networking device to support the new standard. And update all the protocols (DHCP, etc) for more address space. That part is what took a lot of the time for IPv6. Then you have to update all of the software to support 64-bit addresses. Luckily, most of the work was already done for IPv6.
Then you have to support a transition mechanism to talk to IPv4. Except there isn't enough space in new address. IPv6 on the other hand, has enough address space to stuff the IPv4 host and port in the IPv6 address for stateless NAT.