The SFH has domain over their property - and that's where it should reasonably end. Giving them domain over what gets built on everyone else's property is where we get into trouble.
If we choose to overturn those collective preferences, we need to have principled discussions as to why our preferences should rule over the other groups'.
Geometry is an arbitrary basis for power, not a principled one. There’s no deep reason why the block is too small to decide and the state is too big to decide and the City of Sunnyvale is just right.