Personally, I find linear regression makes more sense from an orthogonality of sub-spaces perspective. That the same solution can be derived as the optimum of some cost function is neat though.