What you're suggesting is to set those people in a race to the bottom along with the high school kid and retiree looking for "a little extra money". The point of setting a living wage is to set a bare minimum standard for employment.
You'd also find that the same politicians pushing a for a minimum wage are the ones trying to address broken housing policies, but you'd have to do research instead of just complaining.