I'm sure some can learn just as well on their own as in college, but that is really not the main benefit of it. The main benefit is that the people at your college are self-selecting: they have similar interests as you do, they have similar ambition, they have similar abilities. If this college is a good one, you are going to meet more brilliant, fun, and engaging people in your 4 years than you would in almost any other scenario. Those people you meet will not only make a huge impact on your own development as a person, but will continue to do so long after you are no longer together. They will expose you to a million things you would never be exposed to otherwise. On top of that, the intensity of the experience (from living together, being on campus together, going to parties together, etc.) is simply not comparable to more than a handful of similar options.
Yes, you could stay home, or move elsewhere, and be one of a handful of people like yourself. Or you could go and be surrounded by thousands of brilliant, amazing people for years of the most formative years of your life.
The benefit of going to a good school isn't the classes, it's the people.