I think this is a vast overstatement. As a potential employer, it's in my best interests to have colleges create the best applicants possible, regardless of my personal college status. As a parent, it's in my interest to have any sort of college investment I might make to pay off. As a taxpayer, it's in my best interests to get the best return for our government investment.
To put this entire debate in terms of whether the person speaking has attended college or not is really stretching logic a bit too far. Lots of other reasons to evaluate a college education one way or another.
I think you're walking a fine line here. Especially with this sour grapes remark, it almost seems as if you're saying "these folks didn't get/enjoy college. This is the reason they say what they do. Therefore their opinion is biased and you needn't listen to them."
Sticking somebody in a bucket and then explaining away everything they might say because of the bucket they're in? Sounds to me like one of those logical fallacies everybody loves to quote so much.