Fewer colleges mean:
1) People out of jobs. Teachers will lose their main income and the remaining colleges will have no incentive to increase their payroll because where are they gonna go?
2) Getting into a college will be even more competitive. As companies will still look for people with degrees, colleges can charge even more and candidates to study there will have less choices.
This will only change when NOT studying at a college has no impact in your future income. Or the impact is lesser than today.