f(x) + f(y) = f(x+y)
k*f(x) = f(x*k)
But yeah, in NN nonlinearities are very important, otherwise they would be simplifiable to a single transformation
f(x) = U * V * x,
where U is an n by k matrix and V is a k by m matrix, where k is much smaller than n and m. Basically, we are constraining the set of allowed linear transformations, which is a form of regularization. Convolutional layers in neural networks similarly restrict the allowed linear transformations.
Nevertheless, the power of linear neural networks is far less than that off nonlinear networks.