Adding more levels of cache is a time-honored tradition, and it works.
For example, with the M1, Apple tied the DRAM interface to the CPU. That got them more bandwidth, but then you can't upgrade the memory, which is weak.
But why not do both?
Have e.g. 8GB of high-bandwidth DRAM soldered to the board, essentially that amount of L4 cache, but keep the DIMM slots where you can add arbitrarily more memory with less bandwidth.