My suspicion is that these are pandemic-brain decisions. I think Ada entered mass production at the start of 2022 absolute latest (remember the "TSMC refuses to cancel 5nm wafer order" headlines?) so it would have been specced early/mid 2021 most likely. It's about a year from tapeout to launch typically, and if rumors were true they were ready to launch in june 2022 then they would have taped out in mid-2021 and been in volume late 2021/early 2022.
Everyone in early 2021 was trying to absolutely maximize the number of units shipped, and if you need 4GB more memory per card then you ship 33% less units for your GDDR supply, you get more than 10% less dies per wafer, etc. It would have seemed like a good decision at the time especially with mesh shaders eventually coming in and reducing a bit of the VRAM pressure etc.
As it stands, short of taping out some new dies, NVIDIA really cannot do anything about some of the gaps in the market because their only option is insane cutdowns on AD102 and AD104 to fit the gap. Cutting AD102 down to (rumored) 4080 Super shader count would be a 40% cut. Cutting AD104 down to 4060 Ti shader count would be a 50% cut. That is not yield harvesting, that's throwing away half your die. And that's why a 4080 Super 20GB was absolutely never in the cards once AI started to really take off - NVIDIA can already sell those as Quadro A5000 or A6000 anyway, why would they throw away half of a working chip to make it cheaper for gamers? 4090D is absolutely the final nail in that coffin, it will not happen at this point, every shitty yield they have will go there.
AMD themselves have gone a different route and disambiguated memory from compute. They already put 2 memory PHYs per MCD attached to a single infinity link (so they are "doubling up") which relieves the pressure of GDDR density stalling out at 16gbit for a prolonged (I think unexpectedly) long period of time. It also provides a degree of physical fanout which relieves the routing pressure etc (look at the footprint of an A6000 or RTX 3090, it's crazy how tight they rammed the chips in).
It surprises me that AMD haven't done a 4-PHY MCD. Even if you can't route that many GDDR lanes, you could also do sort of the opposite of the 7900GRE/7900M and put a smaller chip in a larger package but it commands the full memory bus width of a 7900XTX, so you have Radeon Pro 7800 48GB or whatever. It's not without its downsides (RDNA3 idle and low-load power consumption remains atrocious and probably always will due to the link power and due to the caches being on the other side of the link) but they're accidentally sitting on the key to big-VRAM cards at the moment of big-LLM models. What could be more of a push for ROCm then "hey here's a Radeon Pro W7800 48GB for $1499 and we can actually ship them today"?