So I feel like you just agreed with my premise, no?
The value (and thus the tax) on my empty lot with nothing but dirt is going to be set by the tax of the developed lot right next to it, which might contain a highrise apartment building.
So you're saying I will be taxed on the value of an imaginary building I don't actually have, just because the neighboring lot has one. So there's no way I can afford to pay that since the bare dirt doesn't give me any income. So I will lose this land due to the tax.
Am I wrong?