Disclaimer: I am not a historian, my knowledge of Rickover is probably wrong and probably low quality.
I guess you're kind of asking how did Rickover maintain people's intrinsic motivation (sense of autonomy, mastery, and purpose), despite asking a lot from workers particularly in a frequently hostile way?
I think if I had to guess it comes down to two words: "Mission driven."
I think Rickover went all in on purpose. I think every single subordinate had explicit responsibility that mattered. Not only that, but there was the greater context of the cold war. It didn't just matter, it mattered.
My potentially wrong understanding is that Rickover assigned responsibilities, not tasks.
I think Rickover was a pioneer of the technical interview. His interviews were intense and existed to filter out people willing to BS or play games. I think he wanted people who were willing to speak truth to power. I think his interview process, while unconventional, was probably central to his management.
Rickover was a major proponent of (liberal) education (as opposed to technical). He strongly believed in molding quality clay and making experts, rather than hiring experts. That means investment. Investing in a person is validation that they are worth investing in. A person who is invested in probably feels a sense of debt.
> How can people feel ownership while they're being frequently told their output isn't satisfactory?
I think when communicating to someone that their output is unsatisfactory, it can be done in different ways. I think you could tell someone their output is unsatisfactory in a technical way they might even agree with, or you could communicate to someone that they are producing unsatisfactory output. I think that is a night and day distinction. I guess the right question is does the person feel that it is "us vs the problem" or "them vs the manager?" Rickover seemed like a very "us vs the problem" kind of person.
I think some workers also don't get told why their work is unsatisfactory, which is very damaging. Knowing the consequences of unsatisfactory work is centering.
What is clear is that I think Rickover would have debugged the unsatisfactory output, probably himself. Unrealistic time pressure is a failure of management, not a failure of an employee, so I think Rickover would have been upset if a timeline shifted, but only if he was not told early and told why.
My understanding is that it was frequently contractors and people who would fleece the government that would draw his greatest ire.
I think the context of getting yelled at matters, too. Getting a verbal shakedown, especially by a CEO, communicates that what you are working on is important and there are real consequences and that you have the power over them. Yelling means you care. When other people care, you care.
Things like burnout are often thought of as stress issues, when they have very little to do with stress and a lot to do with progress. Feeling stress but not making progress is what creates burnout. Stress with progress is accomplishment and boosts future stress tolerance because the progress ends up justifying the stress.
I don't know if you've heard Gordon Ramsey speak about his abuse, but he makes a distinction between the abuse he experienced from his dad, and the abuse he experienced under Guy Savoy. Guy's abuse was something he valued because it had a purpose, the purpose was to get better food.
> How do you balance firing people who can't perform without creating a culture of fear?
I think if you have a clear ethos about firing vs training this isn't practically a problem. I had co-workers that I wished were fired. Firing those people would have made me respect management as competent. If you fire someone who should be fired I don't think that creates fear. Not firing dangerous co workers might be an even larger crisis.
Arbitrary firing creates fear. A manager firing an employee for what are the managers failures, not the employees, creates fear.
If you read about the NTSB investigation of an ATC issue resulting in a runway plane collision that was posted several weeks ago. I think that's relevant. The mistake the ATC made could have been made by anyone. So firing that ATC would have created a culture of fear because any other ATC could have seen themselves in her position. Boeing has a culture of fear, not because people are fired, but because if you speak up, you suddenly become a person fingers can be pointed at, and therefore a target for legal discovery and firing. The fear comes from the people in power not taking responsibility, not the firing.
If you fire someone, and people can imagine that being them, that's the culture of fear.
Rickover was probably an influence for navy seal leadership which is analyzed in more detail in the very good book Extreme Ownership, which explicitly addresses unsatisfactory workers and firing. If you like Rickover, you'd probably like that book. It is much much better than it looks or sounds.