The point of work like this is to characterize in more detail the kinds of operations transformers are capable of representing. It's in the same spirit as the efforts to figure out the classes of formal languages that transformers can recognize. Nobody is actually suggesting to replace LR parsers or Kalman filters with transformers in applications where those tools are already working well.

Think of what would happen if the result were negative. Like, what if the size of transformer needed to represent the KF grows exponentially with the dimension of the linear system. That would certainly cast doubt on the prospect of using transformers for filtering-like problems. It might also suggest changes to the transformer architecture to fix the issue.

Since the result is positive, our belief that transformers are reasonable for filtering-like problems is strengthened a bit.