I'm not sure if I agree with the premise that the gifted students keep all the resources to themselves. If anything they should require less mentoring, less discipling and their parents should be more likely to be involved financially and in PTA.
And when gifted programs become screeners of affluence then there are going to be financially involved parents in the PTA which benefits the whole school. But if all of those parents end up at the "good" schools then those schools have considerably more funding (on top of the additional funding from the DOE) to allow them to actually educate their students. It is all a self reinforcing cycle that benefits a few and disadvantages many.
Putting limits on how much a parent can help and support their child is a much more difficult ethical and political argument to make
To compare, Stuyvesant High School, the best in NYC, is $18k per student.
The money evaporates before it reaches them.