Sure, you can't build whole from-scratch circuits purely from hardware-store parts, but if you run out of resistors/pots/whatever, you can restock pretty cheaply in any town big enough to have professional electricians operating out of it.
Also, I should probably point out that a lot of places (e.g. Best Buy) still do sell electronic components—they just sell combinations of them in lots and call the result a "hobbyist kit." It's much easier on their shelf space, and their employees, to just carry one SKU that you can point anyone looking for components at, which will give them most of what they might need.
For all the other stuff (what you might think of as a "booster pack" to a hobbyist kit's "base pack"), there's no more economical way to get exactly what you want than ordering online anyway. Expecting there to be a store in every rural town carrying every random IC one might need for a project is just insane, when you can order a pull-sheet of them for $2 no matter where you are.
The prices are, of course, high, but when I break a bolt or lose one for my car or other project, Tacoma Screw has an exact (or better) replacement.
The hardware store sells bolts and nuts, too, but the selection is erratic and the quality is dubious. When I don't want my transmission falling off, I want top quality bolts guaranteed.
I use them for all replacement bolts in my car except for internal engine bolts, where I source them from racing suppliers.
Their physical catalog is one hard catalog that I really miss! Despite the fact that the website undoubtedly offers more product than the multi-thousand paged hard edition, I remember many projects in my college days inspired by simply flipping through it's pages.
Edit: I just clicked the link and realized that is the same place lol, yeah it is awesome.
They don't have very much single-component selection -- although, to be honest, YDI Electronics selection is kind of limited too (understandably because, well, internet).