Comment by _djo_ | Hacker News Reader

Comment by _djo_ | Hacker News Reader

Parent Full thread

The Integrated Navigation and Weapons System (INWS) developed by ATE Aerospace for the Hawk Mk.120 was written entirely in C, was DO-178B-certified, ran to a million lines of code and cost approximately $100 million for the development of the software and hardware combined.

The software includes integration all the standard military navigation and weapons targeting and control functionality, along with a secure datalink using the Link-ZA protocol and a nifty radar simulation system (basically an airborne LAN for air-to-air training) that runs over it.

They just delivered the latest iteration of the software which allows them to receive data-linked air picture information from other aircraft, such as an AWACS platform. Throughout all this the project has had no major snags, has missed no deadlines and has come in on budget.

So my point being that it's possible to write complex safety-critical software in C if your processes and programmers are good enough and you're willing to pay about $100 million for it.

Emphasis especially here:

> if your processes ... are good enough

I would say that it's possible to write complex safety-critical software in any language, provided it's completely specified beforehand in, well, military detail. This program was likely done when that spec hit the programmers' desks--the project from that point on basically consisted of transforming pseudocode to language-of-choice-X without mucking anything up in the process.

Not all languages lend themselves to formal certification, which is why such mission-critical software is typically written only in C, C++ or Ada.

And while the specification process is obviously extremely extensive, it's only a part of the total process and not a guarantee of bug-free code. Many of these projects have gone pear-shaped during the implementation phase despite having good specifications.

So that's why having good programmers, a good development process, solid libraries and great tooling is so important for the implementation phase. Especially because the requirements always change during development, even for mission-critical military avionics software.

One of the main problems with the F-35 program is that the implementation phase for the development of the plane's software has gone badly wrong. It's already by far the most expensive avionics program ever.

Finally the testing phase is super-critical. ATE designed their own extremely thorough testing setup, ranging from software verifiers to full-scale test-benches to simulate on the ground the full range of software functionality in flight.

It's a difficult process that's very easy to get wrong as a result of the incredible complexity of modern avionics and weapons systems. I am in awe of those teams that get it right.

There is nothing about C or C++ which would make programs written in these languages especially suitable for formal verification.

You could argue the one language which actually does lend itself to formal verification is the one which has been formally specified via operational semantics; namely, Standard ML.

There is one thing that helps about C and C++. In a safety-critical system, the entire runtime has to be certified. This isn't a problem for C and C++ because their runtimes are very minimal. DO-178b Level A-certified implementations of the standard libraries exist for various real-time operating systems. As long as you use them, you're golden.

However, any system that runs on top of a virtual machine is problematic because of how extensive their runtimes are. Developing an efficient and competitive DO-178b-certified JVM, for example, would be a huge undertaking, but so would modifying an existing JVM.

Further, anything garbage collected is a problem because you can't have any situations where a timing constraint is violated because of your garbage collector. The best way to prevent this would be to schedule, say, 50 milliseconds every second for garbage collection. But there's still a limit there where the GC can no longer keep up with the rate of garbage production, which could lead to your application running out of memory and failing. Proving that your application will never reach said limit would be hard either by static or dynamic analysis.

I actually once worked on a safety critical military application (though not DO-178b certified), and for a new version, we switched from C to real-time Java (RTSJ). RTSJ actually let's you avoid unpredictable GC altogether through what is called "scoped memory" in your critical threads, and provides real-time scheduling if run on a RTOS.

We switched because we wanted to reduce the risks of bugs, but you must remember that in most mission critical applications, one must consider failure not only in the software. You use redundancy - in the software, in the hardware, and in the network. We even used several separate power systems and several redundant generators.

Interesting, thanks! I didn't know about RTSJ and a cursory search in preparing my earlier comment only popped up links that looked bitrotted, so I assumed it wasn't a widespread technology.

It isn't widespread. Then again, hard real-time systems aren't widespread either. I think Boeing was (maybe still is) using RTSJ. The only downside it has is increased RAM footprint relative to C(which makes it unsuitable for very limited devices) and a small performance hit relative to regular Java because some classes need to be compiled to machine language ahead of time (the Hotspot JIT can do some very nifty optimization at runtime, but those come at a cost of non-determinism. Real-time systems always trade performance for determinism).

EDIT: see http://goedel.cs.uiowa.edu/MVD/talks/Vitek.pdf presentation slides from 9/2012 on realtime and safety-critical Java.

That's exactly the reason why C, C++ and Ada continue to be the primary choices.

There are a few interesting developments regarding Java for safety-critical and mission-critical applications. One is JSR-302, which is defining a safety-critical subset of Java which will not have garbage collection and will use of a run-time stack rather than a heap for temporary object allocation, the removal of dynamic class-loading, changes to task scheduling and a smaller standard library. These should make it possible to certify JSR-302-compatible Java runtimes and applications for safety-critical and mission-critical tasks.

The second is DO-178C, the follow-on to DO-178B, which will incorporate additional work on the certification of object-oriented languages such as Java.

I agree. It depends on what subset of C and C++ is used. If macros are used the only way to actually verify the code will be to pre-process it. Worse is if templates are used. Some templates don't even finish compiling.

http://cpptruths.blogspot.com/2005/11/c-templates-are-turing...

Most likely no one will write such a template in a mission critical system. However, from a formal verification standpoint the only way to verify the code is to check the binaries.

You don't need to look that far afield. Literally everything runs over a kernel written in C.

But that being said: this is just proof that it "can" be done, not that there aren't better choices. Specifically, could you do a missile management system in less than a million lines of code or for less than $100M with equivalent performance and quality on another platform? I think most of our intuitions would agree that you could.

As far as I'm aware, a lot of the safety critical certification requires the entire stack to be certified, in which case the OS (which would probably be a hard-realtime OS) would be certified too.

Probably part of the $100M price tag is a hefty license fee to use said OS.

Yep, unfortunately I don't know which OS they went for but it was one of the commercially-available RTOSes on the market. Those don't come cheap.

Avionics software certification is phenomenally expensive. Both Airbus and Boeing have warned in recent years that it has become such a big part of their development costs that they're concerned about it becoming unsustainable.

Literally?

Meaningless semantic quibble. Yes, "literally" everything. Or everything worth talking about anyway. Do you have a counter example of a complete system seeing active development in the modern world without any use of C at all? I certainly can't think of one.

But having C with all its vulnerabilities in the kernel caused many - if not most - of the security relevant issues of the kernel(s). Think of buffer overflows alone.

Of course, many micros are still programmed in assembly. And assembly still plays a big part in many subsystems, though I would concede much would be written in C. Not literally everything.

I'm aware of no microcontroller work in the modern world that is done solely in machine code. Again, you have a counterexample?

I write assembly for microcontrollers constantly. If you require reliable timing at the cycle level, you have absolutely no choice in the matter.

Sure. But you don't write a system only in assembly. The point, now buried in this silly thread, was alexchamberlain's nit that it wasn't "literally" true that all systems are built on top of C kernels (whether it be an OS kernel, firmware, whatever).

You can call me what you like and if you'd said "everything" instead of "literally everything", I would have let it slide. You were being quite misleading.

Given that you haven't provided a single counterexample, I can't see how I'm being misleading. All modern systems are built on C. All of them.

And for the record, I didn't call you anything. A "nit" is a pedantic correction, c.f "nitpicking". I could have said "quibble" too. It's a reference to your words, not you.

Reply on news.ycombinator.com