Car dealers are important for the same reason a shoe store is important. You've got to go in and try on the shoe first to see if it fits. But much like with shoes, consumers can figure out what fits and then go home and order from someone else.
The only difference is (in my eyes) it would be even worse with cars, because it's easier to spend an extra $5-10 supporting your local shoe store than it is to spend an extra couple thousand supporting the dealer.
Dealers are also a portal for interacting with the manufacturer for things like service, recalls, and TSBs. I'm not sure what, if anything, would step in to fill that void if they disappear.