We had already eliminated Docker from our CI/CD and live deployments (using Kaniko for CI/CD and containerd for live), so we just modified our container build pipeline to run Kaniko twice (once for x86_64 and once for arm64) and then use manifest-tool to build and upload the multi-arch container manifest. I think we had it working within a day, and then it took a week or so to test and validate all our images, and one or two needed some attention due to having dependencies that weren't already multi-arch. Overall it was really painless.
For testing container builds on developer machines most people are either using Docker on macOS which already handles multi-arch cleanly, or using buildah on Linux which also handles multi-arch automatically if you set up qemu binfmt support. So that has been pretty painless too.
I would say if you are doing a lot of horrible workarounds, it's probably time to step back and look at improving the processes (like your pipelines).