The biggest challenge was getting multi-arch builds sorted. Ended up putting together a layer on top of QEMU to run both x86-64 and aarch64 VMs (https://github.com/beringresearch/macpine). Have pre-baked VMs with LXD installed inside each instance, with main software builds taking place inside LXD containers - works pretty well so far.