The parallelism is nice, but it only helps cloud inferencers if there is an efficiency improvement. Autoregressive inference isn't parallel, but parallelism can be recovered with sufficient batching.
Love the idea to treat the sentences like image generation.