Spinning up an ARM machine to build a container that needs to be shipped over the network to a Jetson is a pain when developing.
More expensive, more power consuming? Sure. More sanity? Massively better dev experience? Massively better production/ops experience? Absolutely.
Or just have Jenkins etc trigger the launch of one when your build job needs it.