So on small screens this will make the font even smaller.

This is the wrong solutions.

There is a reason why 16pt is the standard font-size. And its no problem to design a responsive website with columns that use 'between 45 and 75 characters per line'.