At which point it really makes no difference if you physically reside within city limits or live in a suburban/rural area as the time and effort to get to the places that cater to your particular niche approaches being about the same in all cases. In fact, in my experience, it is often easier to access the amenities of a large city when you don't live in it!
Edit: ok I take that back we always used a car to drive ten minutes to tryvann ski mountain in Oslo but that’s because we have jobs and the bus takes like 40 minutes.
There is probably one or two places in the city that cater to that kind of uniqueness, but the chances of it being next door is unlikely. Most likely you'll have to travel long distances to find it. Longer than your feet can reasonably take you. At which point it doesn't matter all that much which direction you are coming from.
What is your reference point here? This isn't the case in any large European, Asian or dense US city like NYC, Boston and Chicago.