You are very clever to point out that people can actually walk around in a suburb, doesn't change the point.
This conversation is by definition about people living on the margins. Similarly, the "vast vast majority" of people do not qualify for SNAP.
Pointing out that most people don't live on the margins is clever, but not particularly relevant, in a discussion about people who live at the very edge of society.
In other words, we have two things, and want to make one the cause of the other, but that's not entirely clear. Maybe the grocery stores die in areas where the population doesn't frequent grocery stores for whatever reason, or maybe people don't frequent grocery stores because they died, or maybe a mix of both.
In public policy the question is generally if something has a measurable impact, and if that impact is worth the investment. Multiple small but very cheap improvements may be better than large but extremely expensive ones.
Food deserts are a symptom of a problem, but the solution isn't necessarily to directly treat that symptom.