Max Crashes Strengthen Resolve of Boeing to Automate Flight
wsj.com
wsj.com
This article[1] linked off of the OP goes into detail about their crashes. It describes a design philosophy that pushes robustness fully onto the pilot, expecting him to flawlessly stitch together independent automated systems and unreasonably rapidly interpret and react to their error modes. In the case described in the article, separate automated systems had contrasting views of the world, and shouted their contradictory errors at the pilots, relying on them to be able to instantly reconcile these models and figure out what the root problem was. (In this case, a specific sensor failure made a single system think the plane was pointed up, and this system made the isolated decision to push the nose downwards while warning that the nose was too high. Naturally, other systems simultaneously complained about the nosedive).
One solution, of course, is to remove automated systems from planes. But assuming they provide any value (I don't think this is in dispute?): another, more reasonable strategy is to commit more fully to an automated system design that is intentionally fault-tolerant, robust to system and sensor errors, and has a more complete view of the plane's world. Instead of requiring the overall state of the meta-system to be lossily communicated to humans in a time-critical emergency, the newer designs are intended to use all the information available to the plane's systems to reconcile subcompoments' contradictory view of the world. Note that this doesn't preclude human control at all: if anything, it makes pilot control _safer_, since the system can provide a coherent view to the pilot in a way that's currently impossible.
I don't know enough about large aircraft operation to fully comment on the wisdom of this approach, but it's substantially more complex than "auto system failed, why would they expand use".
[1] https://www.wsj.com/articles/the-four-second-catastrophe-how...
I disagree, strongly. The root cause of the issue isn't pilot training.
The root fault is a design issue.
Without the existence of MCAS, these crashes would not have occurred.
Why? Because they would never have been built.
Why? Because the 737 MAX 8 design would not have received approval to be incorporated into the 737 type certificate.
Why? Because the MAX 8, without MCAS, has different handling characteristics during certain phases of flight than the original 737.
Why does it handle differently? Because it has larger and more powerful engines, located differently on the wing, and these changes cause the aircraft to want to pull up more when climbing.
I don't know that the MCAS's programmed behaviors have a good analogy, I can't think of one right now.
- The MCAS system. Got bad information and did a very bad thing. It was the entire time operating as it was designed. The fact that the astronauts were not given the information on how to turn HAL off in the event he goes berserk isn't really their fault.
Just to be clear - The MCAS system was actively trying to fly the plane into the ground. It thought it was trying to save the plane from a stall. It was wrong. The pilots attempted to override the system. They could not. They attempted to outfly the system. They could not. They could have killed the system. Boeing did not teach them how.
Holy crap the very first paragraph is opposite-of-reality.txt
> Boeing Co. is increasingly committed to transferring more control of aircraft from pilots to computers after two crashes exposed flaws in an automated system on its 737 MAX that overpowered aviators in the disasters.
The whole problem was the computer overriding the best efforts of the pilots until the planes nosed into the ground.
I would love it when the planes are as easily operated as an elevator. It's probably more realistic than self-driving cars.
It’s the disconnect between pilots mental models of what’s going on and what’s actually going on that caused the Max‘s and several other related crashes on different aircraft. As such software designed based on how the system actually works may actually be both simpler and safer than increasingly abstracted fly by wire systems trying to safely intuit what the pilots want.
An example from 10 years ago: https://www.fastcompany.com/1669720/how-lousy-cockpit-design...
When the FAA certified the 737 Max the MCAS Software was able to move the horizontal tail a max of 0.6 degrees. (out of a total of 5 degrees) The software flying on those planes was in fact able to move the horizontal tail 4.17 times that amount: 2.5 degrees. They never told anyone this changed. The system went from being able to control 12% of the tail range to 50% of its range.
Those pilots were pulling up on the nose of the aircraft with their feet pressed against the controls. Directing the aircraft with clear intent and skill. The 737's automation had unbeknownst to them and with their knowledge of the aircraft - un-overridable, tilted the horizontal tail into a position that would, under any operating condition direct the nose of the craft into the ground.
But it's reasonable to hold Boeing to a higher standard of excellence as an experienced aircraft mfg.
I'd like to learn more about the 31 pages of missing maintenance logs. That's a very big deal.
https://www.npr.org/2019/10/25/773291951/pilots-ground-crew-...
"Clearly, we need to remove the human pilots and double down on the automation."
[This is where the people using shallow analysis go nuts. But wait.]
Because the flawed system was only installed for the purpose of avoiding the expense of retraining human pilots to a new model of aircraft! In a fully-integrated automated piloting system, there is no separate trim-compensating subsystem. The trim parameters are plugged directly into the aircraft model, and the automatic pilot has access to all the sensors in the craft to make its trim decisions, instead of just the angle-of-attack sensor.
Instead of re-certifying hundreds of pilots, you just validate one new aircraft data model.
I'm not sure that's the only reason. Otherwise they should have started building them without MCAS and retraining pilots several months ago. That seems far simpler/cheaper than whatever they're doing now.
Such an aircraft would no longer be considered a 737 for the purposes of pilot certification.
The whole thing was motivated by that. Pretend it's the same plane, so that the pilots can pretend to know how to fly it.
It's analogous to hiring a Java or C# applications software developer when your codebase is actually in a proprietary variant of C++. It's close enough that they think they can code in it, but no one ever mentioned how the memory management subsystem is actually just a library written by Dave, who left the company in 2004, and if you aren't careful to clean up your references correctly (according to Dave), the biweekly analysis run will crash-to-desktop in the 14th hour, and you will be called by a panicked twentysomething at 5 AM on a Saturday. Then you find out about the custom compiler (also Dave), and start sending out resumes before it's too late. But with pilots, there's no one else to send the resume to.
What is painful, is that the pilots recognized that the system was in error, and attempted to correct it, but were unaware that it had the capability to override them. They were unaware because the designers intentionally hid the mechanism. That is made the mechanism hard to see, understand, and needing special knowledge to disable. The designers had not considered all failure modes, but acted as if their implementation was failure-proof and never to be tampered with.
The question in this case is how can one verify something like an automated aircraft system? And more importantly, if there is a technique or practice to assure the system is valid, is the company trying to build it mature enough in its engineering practices to follow it properly?
It was engineered like it was version 0.1 beta. When you realize that the -MAX crashes were very close to single fault accidents it's even more crazy. Normally to crash an airplane you need to thread the needle through multiple failure states thanks to redundancy and safe design. Simple single issue crashes are almost unheard of in the modern era thanks to decades of safety engineering. Aggressive cost engineering is returning us to the perilous air travel of the 40s and 50s.
Now every new car needs to have adaptive cruise control and anti-lock brakes and stability control and Apple CarPlay and over-the-air updates and who knows what else, so there are a ton of things that are hopefully well-behaved black boxes that people hopefully understand mostly correctly, but who knows what could go wrong in a rogue OTA update or something.
I'm not all that familiar with the aerospace industry, but this feels the same. Of course all of these components will be tested, but if no one understands the whole package, who knows what fearsome edge case awaits when Flight Safety Program 32 gets in a fight with Flap Threshold Control Program 3 or whatever.
I guess there's just something more "understandable" about a plane crashing due to an engine failure rather than software fighting the pilot. Or it's more understandable when someone died in a car crash because they couldn't control a skid rather than Autopilot driving into a concrete barrier it avoided previously because something inscrutable in the programming changed.
Perhaps we're heading towards a world with on average better, but less well-understood, outcomes.
We wouldn't have a problem with the software not recognising barriers or mistake road lines correctly as long as this software doesn't claim to be able to handle high-level decisions and fail on something extremely obvious to humans leading to great damages.
Have they literally gone insane?
One solution, of course, is to remove automated systems from planes. But assuming they provide any value (I don't think this is in dispute?): another, more reasonable strategy is to commit more fully to an automated system design that is intentionally fault-tolerant, robust to system and sensor errors, and has a more complete view of the plane's world. Instead of requiring the overall state of the meta-system to be lossily communicated to humans in a time-critical emergency, the newer designs are intended to use all the information available to the plane's systems to reconcile subcompoments' contradictory view of the world. Note that this doesn't preclude human control at all: if anything, it makes pilot control _safer_, since the system can provide a coherent view to the pilot in a way that's currently impossible.
I don't know enough about large aircraft operation to fully comment on the wisdom of this approach, but it's substantially more complex than "auto system failed, why would they expand use".
[1] https://www.wsj.com/articles/the-four-second-catastrophe-how...
Anyway, in response to your comment: perhaps I'm conflating these, but a commitment to robustness seems like it's included in the newer philosophy? It just extends that notion beyond the sensor-level to the system level.
Sure, you can frame an inadequate emphasis on robustness as a "management problem" instead of a technical one, but that seems equally applicable to the "management solution" being put forth, no?
The only thing that would be an appropriate counterweight is if the engineering staff refuses to put a defective design into production. A robust engineering process can make violations and exceptions clear, but if the staff still does what management says, it won't help.
I think Boeing and Airbus have such long delays between project initiation and delivery that planning now to deliver a fully automated flight system in 10 years is likely in the realm of reasonable?
The fundamental problem was that they had new requirements and instead of allowing the engineers to create a new design, management wanted to save money so insisted the engineers shoehorn the new requirements into the old design with hacks resulting in a plane that was fundamentally unstable and ending with hundreds of people dying.
The solution to that problem is to have several executives from Boeing and the FAA in prison. Then break Boeing down and rebuild it as an engineering organization.
After that, yes, continue increasing the automation.