If land is in a city, that gives it a high value and revenue earning potential due to no work or expenditure from the land owner, but ongoing costs to the taxpayer to provide the infrastructure and services that give it this value and potential. Roads, schools, hospitals, etc. these are very costly to the public but increase the value of nearby land.
The land value tax gives the owner an incentive to invest in improvements and get the maximum economic return on the land, and dissincentivising sitting on land in order to benefit 'for free' from it's improving value. So it's about making sure the public gets the maximum benefit from it's investments that improve the value of the land. Taxing improvements directly would have the opposite effect.