I only use Docker for a compilation environment for Yocto, luckily I was able to find someone who had instructions for recompiling the image to aarch64 which runs super fast and efficiently.

It was horrible trying to run emulated, but I was sorting this out using the older version before Rosetta 2 was an option - does that make any difference?

This ^ - use a 32gb model and regularly run out of memory