I mean couldn't you do the same thing in a property tax where the buildings are exempted (ie, a Land value tax)?
The purpose of a LVT is to capture as much property value for the government as possible. Anyone advocating for it is unlikely to recommend exemptions that would interfere with that purpose.