One thing I would find useful in the situation is a shared IPv4 machine that will do host-based HTTP(S) proxying. This way, I can have an AAAA directly to my IPv6 and an A pointing towards the shared machine for those who can only do IPv4.
So even IPv4 only clients can reach your domains