I think that's the jib though - in a lot of countries you don't really "own" the land. You and the government(s) get to just agree that it's your responsibility for a while.
LVT cleans up this incentive some by putting taxes in place in areas where the land is more valuable. I think Georgists would argue for doing that _and_ removing property taxes, specifically so you aren't taxed for the things you own.