The web has physical, font-relative, and viewport-relative lengths which all serve slightly different purposes. Like you, I find font-relative to be super useful much of the time, but the others have their place as well. I just want to point out that CSS specifically doesn't have a resolution-dependent measurement.
Using pixel (1/96 in) sizes for a font is fine if you care about the physical size of the font being displayed. Starting from the system default font-size as you suggest will also work, but you'll need to test it on all the platforms (especially if you use a custom font).