This isn't a transformer, it's a diffusion model. You can't split diffusion models across compute nodes.