People keep suggesting we could've somehow gone half-way, but it's an illogical proposition.
Router ASICs are keyed for IPv4 - that is the only thing they will ever do. Change anything about IPv4, and you have to replace all the routers (heavy duty ones, which do the heavy lifting of the internet).
There is no half-way solution to this problem that is somehow "easier" then existing transition mechanisms.
That's a broad oversimplification. Use the Cisco 6500 line as an example: it's a platform that's very long in the tooth, and through modularity is still functioning quite well, even for IPv6.
Also: let's not pretend that similar issues weren't hit with v4, eg, the routing table explosion.
You can change plenty of things without fork-lifting entire platforms, and serious long-term challenges exist even without considering v6.
You can't "just change the address length". They're hardware ICs. If the address is not 32 bits long, then its a malformed packet that should be ignored. They cannot handle it differently. They can't be reprogrammed.
There's been some consternation over Cisco stuff in this capacity, since they use an ASIC for IPv4 and a software-mechanism for IPv6 which is not nearly as fast.
The cost and complexity of changing hardware is enormous compared to every other part of the process. IPv6 in software has been solved, and where it can't be solved you can shim-it trivially by comparison. Software is not what's holding it back.
The internet goes to some lengths to avoid routing loops as well, so once a packet is passed one router we're not going to be able to easily spin it around and send it back to get to the right place. The net effect is going to be to randomly DDoS a bunch of devices which happen to be behind non IPv4.5 hardware, and so receive 50% of the internets traffic which doesn't get routed properly by an upstream router.