Hacker News Reader

Comment by jpollock | Hacker News Reader

Doesn't that depend on the testing schedule?

If the schedule called for simulations to be run in parallel with the live test, then it's an expected outcome. It should be _expected_ that every test will find a problem. Since this was uncrewed, there was no risk (other than to the uncompleted tests possibly requiring a second flight) to running them in parallel and porting across fixes for any problems that were found.

It was a schedule compression attempt with a cost of second test flight risk if it failed.

View on HN

Boeing took the "we'll do very rigorous engineering up front and prove everything on paper" approach, where SpaceX took the "we'll prove it works by actually launching it" approach (which isn't to say SpaceX isn't operating with engineering rigor).

In theory Boeing ran all their simulations over the past couple years, and this flight should have just been a formality. As it turns out, Boeing is running into a lot of issues when they actually test their hardware.

The problem with simulations and paperwork with high tech engineering is you need enough competent, independent reviewers that understand how the whole system works together.

That sort of thing is rarely organized. So we test it instead.

This is likely an oversimplification. There are multiple organizations, even within Boeing Defense & Space, that write their own flight software. All stovepiped and largely working in parallel. This doesn't even include the commercial folks, infamous for the 737 Max. My understanding is that the St. Louis teams are better regarded, and the folks that worked on DARPA HACMS deserve some credit, but they seem to be outliers. Boeing's culture doesn't seem to prioritize modern software development methods or software rigor on the whole. Functional testing should be the last layer of bug-hunting techniques, not the first or primary. The issue seen on their capsule didn't surprise me at all. Other BDS software groups use utterly outdated software development methods and we should all be a little bit worried.

Obviously it's a simplification; comparing and contrasting the approaches of SpaceX and Boeing would require several walls of text...

At the core of the issue though, the process that SpaceX pitched to NASA (and NASA approved) involved quite a bit of actual hardware testing. Boeing's plan (also approved by NASA) relied much more heavily on simulation, modelling, and other sorts of process validation. For instance, Boeing did not perform an in-flight abort.

It kind-of depends on what sort of issue you're finding in your test. Complex systems like spacecraft often have unexpected interaction effects which only testing can reveal. I would call these the 'good' kind of test learnings - ones that an army of great engineers wouldn't be able to predict.

This is editorializing but it looks like Boeing didn't uncover very complicated interactions - they failed at a more basic level of competency - timer synchronization for the launch issue and then a major software bug for an important orbital maneuver. Those sorts of issues really should be sorted out on the ground using hardware simulators. Furthermore, the timer synchronization failure prevented testing of the docking hardware, further delaying the overall program.

For a human rated vehicle, personally I think you should have at least one full-up, fully nominal test before you send anyone along for the ride.

Reply on news.ycombinator.com