Wouldn't we rather trend model memory requirements down? This way models/devices meet in the middle.
Take almost any given model that fits in X memory pool for some task. Double that, and everything gets better. Training quality goes up, inference quality potentially goes up, you gain the capacity for more batching which dramatically increases performance. Maybe more caching, depending on the app. You can add more models to the pipeline without much fuss.
But the big models that are in fashion now... GPUs are basically always memory capacity limited to some extent.