Convolution is a linear operator (with a certain special structure). So convolutions are represented just like any matrix-vector multiplication. Given the special structure of the convolution operator, it can be represented in a sparse manner, and what's more, the actual computation can also be implemented in an efficient manner, compared to a naive matrix-vector multiplication.

TL;DR: It can be represented easily using Einstein notation. Einstein notation just does not capture the sparsity properties we like; it represents the transformation properties quite nicely.

The smart-aleck answer would be "with a Fourier transformation". But yeah, I think you have a point.