I think I'm in your camp. I left the research world after finishing a PhD, but before I did a post-doc. The first few years of grad school were great fun -- far more than any job I've ever had -- but eventually the music stops, and you need to find a chair. And that's when it starts to suck. The academic job market is dismal, and third-tier universities all over the country are staffed with graduates of top-tier PhD programs. You have to be a true exception to land a good professorship these days, and the ivory tower is littered with the corpses of the brilliant people who failed to reach the top.
I had always been eager to become an entrepreneur, but back when I was right out of college, there was no such thing as YC, and definitely no such thing as a college kid getting venture capital to try out an idea. So I went to grad school. Years later, I'm looking down the barrel of a crappy academic job market, but the startup world had become much more vibrant. So it was compelling.
That said, there's absolutely no way that your average startup is going to be as intellectually fulfilling as graduate studies in a field you love. Most startups are doing boring CRUD apps that take little technical ability or knowledge to produce, but a good grad program in computer science will let you work on the technology that startups will be hyping in 2032. There's no comparison.