Why would it be just "poor non-graduates" subsidising "middle class kids"?
An potential extra tax burden can be allocated from wherever the government sees fit. Like a digital services tax or closing loopholes that are conveniently left open for global corporations.
Do you worry that the healthy poor people are subsidising the middle class sick ones through the NHS? Or that poor careful drivers are subsidising the cost of accidents for the middle class careless ones via their insurance company?