Sounds like OP's spouse was the tipping point. If they stayed on the same page, I wonder if things would've gone differently (ignoring the medical issues which aren't trivial, of course).

The OP seems content to live a life without work. For him, by far the main reason to work was for income. If you don't want to work, then why keep doing so when you no longer need income to fuel your lifestyle? Obviously people are different here, and many (esp. on HN) derive other values from work.

The wrench comes when you're surrounded by people who aren't on the same page. Like OP pointed out, you become more disconnected as you both can't relate to the other's chosen lifestyle. OP sees their peers' lifestyles as consumption-focused. And I'd bet those friends couldn't relate to OP either.

OP's spouse probably felt caught in the middle between freedom and social inclusion. If you can't relate to your peers, early retirement could feel more like stagnation than freedom. Social inclusion is a hard thing to give up.