Speculation on construction and development is absolutely fine and maybe even good! I don't have a problem with a bit of rent-seeking if you went ahead and built something.

However... speculation on the underlying land, which is not fungible and one of the few things on this planet earth that is actually completely inelastic with a totally fixed supply, is bad. I won't say this is a popular tax, but it's not unheard of.

I think a really great example here is Singapore, which effectively doesn't have private land ownership and has been pretty spectacularly well developed in the last few decades.