It's certainly not always possible if you need real dress clothing or real hiking/camping gear. (I've never gotten to the point where I can do a month-long trip that includes a long distance week-long+ walk along with some fairly formal evening wear in a carry-on.)
But you can probably get closer than you think. By myself, if I'm mostly just traveling in cities with "business casual" as dressy as it gets, I can travel almost indefinitely with a 40L travel backpack.
The last trip I really felt I overpacked for with respect to train/transit was some time in London, followed by a long-distance walk where I really needed everything I had with me, followed by an ocean liner return where I needed jackets and real dress shoes--which, as you suggest, even Rockports aren't really general purpose. You always want more shoes than you can reasonably carry.
Wheeled vs. non-wheeled luggage is definitely a tradeoff. I lean towards non-wheeled in general and just maybe take a few more taxis if the luggage is too heavy to schlep around the city a bit.