This was not my experience. Granted, it was a while ago, and I'm not sure I'd call myself "gifted," but being ahead of the rest of the class certainly didn't get you any time to do interesting things -- it got you a pile of busywork and angry glances from the teacher for being such a pain in the ass.
I doubt things have changed much; if anything, due to increased student:teacher ratios, I suspect they've gotten worse. I don't have any kids but if I did, I'd be far more concerned about a particularly bright one being wrecked by public school than I would a special-needs one who lagged the rest of the class.
Class size seems to be the single biggest concern related to education quality. However, it's not entirely clear that smaller classes help; especially not that they help "normal" students (as opposed to those who are "at-risk").
It seems that other things being equal, this is can be a big help to those that need extra help, especially when they're not getting much support educationally at home. But it doesn't seem to have the same effect on kids that are doing OK anyway.
Also, having smaller classes means having more teachers. And that means either offering much higher salaries or settling for less-good teachers. The latter is probably not helpful, and higher salaries may cost money that could deliver greater results when invested in other ways.
Simply assuming that your kids will do better if the classes are smaller is erroneous. Read a bit more here: http://www.edweek.org/rc/issues/class-size/
And more importantly, because leaving gifted kids in unchallenging classes often causes them to burn out and get turned off on school and learning entirely. I've seen it happen, and it's so sad.
Being ahead of the curve in public schools doesn't mean that you get to skip out on classes, you have no greater free time than any of the kids struggling with the material (except perhaps it takes you less time to do homework).
Best case scenario is that unchallenged gifted students take advantage of the most advanced classes available and learn on their own at home. Worst case scenario is that they become jaded and disenchanted with school and learning, stop doing school work because it's beneath their capability, and act out in worse ways.
Lucky for me, some time around the 8th grade or so I turned from the second type of student (falling behind, increasingly disengaged) to the first type (being recognized as one of a handful of top students in my senior class). I was lucky enough to attend an excellent public high-school with good teachers and lots of advanced classes, if those classes and teachers would not have been available to me I don't know for sure what I would have done.