It probably does have something to do with school size, perhaps my class wasn't big enough to support the distinct cliques required (graduating class ~20).
Edit: to add this perspective though, my wife went to school in a relatively small town (Rolla, MO) and also had a 400 or so graduating class, but she claims ignorance of any sort of nerd bullying and thought it was a TV trope only thing too. She graduated in 2002. Maybe something changed between those dates?
Admittedly, it was unrepresentative in other ways: one third Ashkenazi Jewish, one third Persian Jewish, one third mostly-white-and-Asian other, very high population of affluent immigrants. We were plugged in to American pop culture, but I think we were isolated in some senses. Little to no idolization of the sports teams, for example.