The extension headers are fine, because they frequently are not needed, so the fact that they are optional allows the IPv6 header to be significantly simpler than the IPv4 header, which has some seldom used fields which are always present.
As a consequence, IPv6 with extension headers are just dropped by routers in the public Internet.
With one exception, extension headers are not examined or processed
by any node along a packet's delivery path, until the packet reaches
the node (or each of the set of nodes, in the case of multicast)
identified in the Destination Address field of the IPv6 header.
The issue is that the extension headers can be used for nefarious purposes, such as hop-by-hop routing. So your fast hardware-based border routers might be EH-agnostic and transmit packets to your core network. And then a helpful Linux box that implements all of IPv6 in software would reflect the traffic right past your firewall.
Implementing the EHs in hardware is expensive, as you need to do conditional checks. So routers just drop them.
How else would you do it? Forcing every packet to reserve space for every possible option (including all future ones) seems obviously worse.