Then it's not enough to say it, show the numbers. Considering the tiny demographic it serves, I'd imagine the cuts on the basis of inequality are a convenient excuse to slash spending, and not increase it otherwise.
There's no relationship between gutting gifted programs and either alleviating inequality or improving outcomes of all other students. If you want to do the latter, that requires it's own intervention.
This basically apes the rhetoric of the right-wing on the part of social spending. "We can't afford it, it should be spent on other things". Same mentality.