Now scale that up to a number which actually makes any difference and your SUV argument falls appart.
And yes it's totally valid as well to not allow it at all if only a handful of other people can benefit from it.
We are the people and I don't care if a rich person wants to do that. My airspace is more important than that one rich person.