That's true. Most ML algorithms have the iterate-until-converge pattern. How about some tasks like hyper parameter tuning or trying out different algorithms against the same data set? Those can be run in parallel.
ML research should aim to produce more parallel algorithms.