That's actually an important point which gets skipped over frequently. Often these time/distance comparisons are done as if commoners are traveling across entire continents frequently. Except your normal commoner is maybe a couple miles from where they want to go. That leaving your local area makes you "unusual" at the very least.
Fantasy and recreationism often have that issue, where they're still looking at the situation from the prospective of humans who can "relatively" easily travel from America to Europe in a day, and where large portions of the population regularly travel 100's, perhaps 1000's of miles for Holidays or similar reasons.