The first time you quit your job just to not be working is pretty liberating. I do this all the time; I work for 2-3 years and take a year or so off. I'm in the middle of a year off right now. Of course, I'd love to be rich enough to come and go as I please, or have a passive source of income, but instead I save money and take off when I have enough. I've been referring to it as "retiring one year at a time".
There are downsides, of course. It's a lot harder to do if you've got family to support and a mortgage to pay, but I don't have either. And I'm absolutely sacrificing a "real" retirement in the process; I don't have anywhere near the retirement savings that I'd have if I'd worked straight through. But I also simply couldn't have worked straight through; I'd have gone insane by now.
I'm already jealous of my friends who've managed to retire in their 40s and 50s. But they've been jealous of me for decades of travel and freedom.