Because dimension is about the connectivity structure of a set of objects, not the size of the set of objects.

Sets with the same cardinality can have wildly different topologies, which describe the possible paths through the space, which in turn describes the nature of functions into and out of that space, as well as transformations and relationships within that space.

A line through the center of a sphere doesn't necessarily touch the sphere in a "4 dimensional" connectivity structure, must touch the sphere in a "3 dimensional" connectivity structure, while a sphere doesn't even exist in a "1 dimensional" connectivity structure.

The main reason that we use "4 dimensional" spacetimes for particle physics is that there are three independent components of motion with respect to time, and so it makes the math simpler to allow for independent variation of those parameters to functions, instead of compressing them into a single encoded number, eg, derivatives wouldn't function correctly (or likely even be defined) on a single parameter model. An additional reason that we don't use a "1 dimensional" spacetime is because the product of a "1 dimensional" time and a "1 dimensional" space is a "2 dimensional" spacetime -- and we don't have any models of reality that lack either space or time. (Which is good: any model needs to explain both, if only as an emergent feature.)

Topology is about connectivity, not scale; dimension is a topological fact.

The other commenters are making some very good points about how fundamental dimensionality is in studying vector spaces.

Just wanted to add that if you'd like to take a more set-theoretic approach to the topic of dimensionality, you might be interested in Matroid theory, which is a generalization of the notion of linear independence in linear algebra.

I found this video to be helpful in getting a basic sense for what Matroids are: https://www.youtube.com/watch?v=FKTPFy-MGp0

Because you can't describe any system with just 1 number. You are arbitrarily choosing to only pay attention to the cardinality of a set. If you are discussing a vector space, there is more structure than that. This is what the study of linear algebra is all about. The amount of structure you require depends on the question you are asking. Not all questions equate to finding the size of a set.

You are like a blind man feeling the tail of an elephant and claiming the elephant's nature is the same as a rope!

Say you convert color of the sock in the drawer to a range [0,1] on your one and only coordinate. Then you convert dirtiness of the sock to a similar range and just add them together. Can you differentiate between a sock that is 0.4 on color and 0.6 on dirtiness from another sock which is 0.6 on color and 0.4 on dirtiness?

I suppose you could encode it all to one number so that structure is preserved, but dimensions themselves are not going anywhere, they are in your choice of encoding that must reside outside of your one coordinate.