That's 16GB of DRAM. Caches are SRAM, which has a very different set of design tradeoffs. SRAM cells use more transistors so they can't be packed as tightly as DRAM.
If 32mb would be too hard they could have easily went for 1mb.
But they didn’t and that’s a pretty good indicator it doesn’t make a lot of difference.