It's been almost 8 years since I was there. Working on flight software was probably fairly unique compared to other parts of the company. We were critical path for the company more often than not, so there was a lot of visibility and support, and a lot of praise when we hit milestones. Other engineering disciplines often worked harder than us, but didn't get the same praise due to less visibility. Folk at SpaceX are allowed and encouraged to work unsustainably hard, though, and so there's lots of depression leading to burn-out. My manager cared about my well being and it was possible for me to strike a sustainable balance; I had to consciously prioritize it though. I liked most of my coworkers and I'm pretty laid back socially, but a few drove me crazy. We'd play lan games on occasion. The froyo and eventual coffee bar and cafeteria were nice, and most of the company parties at the time were over-the-top. The first 20 seconds of a rocket/spaceship launch that succeeds or fails as a direct result of your work is a notable high.
I wouldn't say I know Elon Musk, but I attended meetings with him on occasion, and happened to sit next to him once at a fancy team dinner. I never directly had a negative experience with him, and in all the outrageous stories that I heard both sides to, he seemed to be acting rationally. He's definitely an intense, awkward dude. If there's a problem he thinks he can solve, he solves it; if you say a problem can't be solved and you don't have hard physical evidence, he'll fire you. As long as you act in good faith, he seems willing to move mountains for you if it will help reach the company goal. He has flaws like any human/martian hybrid, so I imagine some of his detractors are right about some things and some of his proponents are wrong about some things. I certainly wouldn't bet against him though.