Spoken like someone who doesn't have to wear reading glasses. Us old folks will be fighting this one. ;-)
There is some issue with how this should interact with certain things (like touch distance). Should two buttons that are 2cm apart be 4cm or 2cm apart after scaling by a factor of two? There is a fringe case of a display-based ruler, that essentially shouldn't zoom.
Given that we can't support actual-size rulers today, I think we've demonstrated that we can live without them.
P.S. for low resolution displays it's obvious that clear text and graphics is more important than absolute size.
We don't change the definition of a mile based on the type of vehicle travelling it, and we shouldn't change the definition of a point based on the type of display viewing it.