A nice and interesting article, but describing the squared error as "merely convenient" is a bit misleading, given that most noise is Gaussian, due to the central limit theorem.
On the contrary, squared errors are part of the formulas taken as axioms from which to derive the theorem.
I wouldn't be surprised if we could derive a theorem whose essence is tantamount to the essence of the famous Central Limit Theorem, using some other arbitrary formula. Like error to the fourth power.