This is also my sentiment.
I am saving up to retire early. If I mention this to friends, most look at me with big eyes and ask “But what will you spend your day on then!?” in a sceptical tone.
I imagine they think I want to drink beers and play golf all day every day, or something like that.
I’m a bit heart broken, that so many of my friends cannot imagine being masters of their own time, without thinking it would be bad for them and/or unproductive.