It's a huge cost to the whole community, if by the "whole community" you mean the entire US.

Let's say that Apple pays municipal tax of $X-$Y to Austin because of incentive $Y instead of $X to Denver (for example)

Then Denver loses $X while Austin gains $X-$Y and Apple gains $Y. Sure, the Denver loss is only theoretical but Apple has to build somewhere so American municipal taxpayers lose as a group. A "race to the bottom" has real costs.

The municipal governments are behaving rationally; it's the federal government who represents all municipalities as a collective; they're the ones who should be passing a law banning this type of behavior.