I've heard the same stories from serious applicants to the astronaut program. A major problem they cite: living in Houston for N years, doing training, waiting to fly.
The remaining astronauts get stuck with jobs such as "Lunch with an Astronaut".[1] It looks like none of the astronauts from the class of 2013 have been in space yet. Being an ex-astronaut is worse. A few years ago, NASA pulled ex-astronaut passes to NASA Houston, so they can't hang around there after quitting.
[1] https://www.kennedyspacecenter.com/things-to-do/lunch-with-a...