How much do we need to progress before it becomes comparable in terms of energy to the (often already rather energy-inefficient) data pipelines we've been using so far?

Recall that while the cost per token may decrease, CoT multiplies the number of tokens by several orders of magnitude.

They can do it, they're just slower, less reliable and orders of magnitude more energy-expensive.

But yes, they're potentially easier to setup.