You are correct. It is possible to construct a linear neural network, where an algebraic function could be extracted, but in practice, almost all networks use non-linear activation functions.
In the case of a linear network, the function would be a dot product between the input and the weights:
๐ฅ1๐ค1+๐ฅ2๐ค2+๐ฅ3๐ค3 ... for all inputs (xi) and weights (wi)