Financial is the least of it. There is a spike of people who die right after they retire - they never learned how to fill their free time. Keeping busy and having friends is one of the keys to live a long life. A job provides both of those for you.
source?