You went through Axler right?

It's a respectable idea that determinants should not be as important as they are on a pedagogical point of view imo. But they can be extremely useful.

From my point of view, matrices are a bad abstraction in general. For example if we have a linear or a bilinear form, both can be written as matrices of the same size, we can no more differentiate the two objects.

The trouble comes when we want to apply some transformation over those matrices.. as they do not use the same transformations laws.. Namely A^-1.L.A and A^T.B.A

This is a very usual error in computer graphics when we want to make some transformation on a 4x4 matrix and we do not know which kind of object it is.