Have you tried something like 1024x768? Going to full 1024x1024 would double your VRAM usage so I can see why that wouldn’t work.

For my uses, the real benefit of having more VRAM is that you can generate more images simultaneously. My 3080 can generate only one 512x512 in 7 seconds but three 384x384 in that same timeframe. It’s allowed me to generate grids of hundreds of images in just a few minutes.