If anything it's not extreme enough. Consider how much area in city downtowns is given over to paved-over parking lots that are just left to sit there rent-keeping for years. That's worse than just dirt fields, because at least the dirt fields can support life!
> My concern is that the same incentive pushing one to sell a pile of dirt would exist for someone owning an undeveloped, natural plot of land.
...if the taxes are high enough to incentivize them to sell.
If you have an undeveloped plot of land in a city where land taxes are high, you should be incentivized to either sell that to someone who will make use of it, or donate it to the city to serve as a public park, rather than getting to have your own private green space somewhere that space is at a premium for everyone else.