imo this comes down to centralization and the popularity of "big box" stores. People generally don't want to go to a specialty store anymore, the toy section at Walmart is fine.
It's likely the specialty stores are being crushed by online retailers. Their raison d'être was always having more inventory depth than general retail, e.g. the shoe store has a much better selection than the shoe department at Sears. But if you're after selection, you can't do better than your browser.
But I’m definitely a purposeful shopper: I’m always in and out with one goal in mind, I never “browse”. Maybe wal-mart and the like are more attractive to people who just kinda want to browse around and kill time?