FWIW, the various diffusion models (tend to) use cross attention to an attention-based approach.
Reply on news.ycombinator.com