This is not true in cities like NYC or Boston; also likely not true in many parts of LA. This is why people tear down houses to build new houses.
Looking at Vancouver for an example it’s easy to see how often the houses themselves are insignificant when compared to the value of the underlying land.
The tax system allowed a developer to take a 30-year speculative bet that eventually they would build their skyscraper. Their plans almost completely fell through the year before construction started because they were planning an even bigger building than they built but couldn't lock up enough tenants. They settled on a 40-story, 1 million square foot tower that stands there today and dwarfs all of the buildings at its feet.
However, I can see a long-term venture fund taking the exact bet you suggested. If the value of the new building that combines parcels is high enough, there is no way a land value tax would address your desire to stop landlords from waiting for opportune development projects.
What you would like is a punitive vacancy tax, perhaps combined with a required sale after a certain amount of vacancy.
My ultimate question is why you might want this. A land value tax is supposed to encourage the highest and best use of a parcel. This developer recognized that the highest and best use of a parcel might actually include the use of adjacent parcels. Shouldn't we encourage this?