Hacker News Reader

Comment by danjayh | Hacker News Reader

4 years is a big lag if you're not doing safety critical work. If you are doing safety critical, 4 years isn't particularly fast, but it's not unreasonable. If you've never worked in aerospace or automotive, you would be shocked at the requirements for software and hardware that can kill people if it breaks (assuming that Tesla is following the automotive equivalent of DO-178/DO-254, ISO26262)... I guess the automotive guys do have an advantage because they're not required to have independence for the various stages of verification, but it's still a big chunk of work (more than the actual development).

View on HN

Everything we know about how Tesla operates as a company indicates that they do not freeze development of a technology or component and then spend four years testing and verifying it before deployment.

That's not exactly how it works. Development doesn't freeze - change requests, PRs, etc. can all be rolled in right up to the end and even after release if the appropriate process is followed, and in software, even in a safety critical world, you can pipeline updates out in a reasonably fast manner (we can get it down to about two weeks for minor updates). Hardware is an entirely different beast. You have to do FPGA development and prove-out, gate-level sims, timing accurate sims, thousands and thousands and thousands of simulated testbenches (hardware sim is orders of magnitude slower than software sim - hours per second, even on the most advanced equipment). Therefore, each testbench must be developed to test a very specific thing - in general, you can't just run your high-level software unit tests. Anyway, even if Tesla is the kind of company to roll software (doable) or boards (doable, but harder) right up to the last minute ... they do not (and neither does anybody else) spin a chip over and over again during the course of the development cycle. Speaking as the lead of a SW team that proved out a safety-critical custom SOC, it is colossally expensive. Every single iteration takes many months and millions of dollars.

You're agreeing with me. Tesla in all likelihood does not freeze development of a chip model for 4 years before releasing it. So you cannot compare Tesla's current chip with a Google chip that is 4 years old.

please read to the end nobody iterates chips right up to release. That's a software thing.

> If you are doing safety critical

Well, Tesla clearly isn't doing safety critical work on autonomous systems (it wouldn't have even launched "autopilot" if it was), so 4 years is indeed a big lag.

Even if it's not now, their intention is to provide higher levels of SAE automation on the same software as time progresses, which means that the current hardware probably has to hit a higher design assurance level than the current software.

Sure, but intentions are irrelevant. What matters is what happens.

Tesla claims [1] that autopilot driving is safer than regular driving in terms of risk of accident per mile. Not sure if I agree with the way they've constructed their statistics, but for a moment take them at face value.

Isn't then the safer thing to release and encourage autopilot to the largest extent possible?

[1] https://www.tesla.com/blog/q3-2018-vehicle-safety-report?red...

That statistic is wildly and purposefully misleading. It compares the safety of Autopilot (which only drives on the freeway) with human drivers on any road. Accidents-per-mile are already much lower on the freeway.

Plus, the statistic ignores the fact that responsibility is actually shared between humans and the car. If autopilot gets into an unsafe situation and gives control to the human at the last moment, the car company might claim that the accident happened under human control. We have a long way to go before we can truly understand the safety of semi-autonomous vehicles.

I'd even expect their human driven statistics not actually representative or useful for comparisons. Their cars are expensive, mostly excluding young and thus often unexperienced drivers. They also don't particularly target older people, the other group with high accident rates.

Controlling for all those variables is hard and not to their benefit, thus I'd never expect their pr team to do so. But I wonder if that might even creep into their automated driving stats... e.g. by drivers taking control in dangerous situations.

>> If autopilot gets into an unsafe situation and gives control to the human at the last moment

We have lots of experience with the reverse, where the human drives but the autopilot rips control away when things go wrong. Traction control, radar-triggered braking, even basic ABS is a machine taking over from the human once things get hairy. Several AirForce fighters have systems that will sometimes ignore the pilot's inputs and save the aircraft from a crash. That's the real path for semi-autonomous cars imho: you do all the driving until the robot steps in to save your ass.

(I cannot wait for the day someone tries to sell a car that will automatically obey speed limits regardless of driver input. That's a stock to short.)

Well, it’s not autopilot, it’s guided steering and gas, which on a freeway is the “easy” autopilot path: trivially (for a human) identifiable routes, signage, infrequent low visibility occluded by tight turns, etc. I’d guess a) if you removed the driver from behind the wheel the accident per mile would skyrocket and b) humans already have a much improved accident rate on the freeway compared to general purpose driving. Is it better than a driver driving alone? Maybe—time will tell as more dose off behind wheel, although I am hopeful—but it’s hardly designed to operate without failure and should not be referred to as autopilot at all. It would help if Tesla provided a meaningful comparison instead of the disingenuous one they trot around.

Safety improvements on divided highways I think are to the point that driving is so boring for humans that any increase in safty ends up with offsetting human behaviors like, eating, texting, phone calls, looking at the scenery, falling asleep. If auto-cars can do on-ramp to off-ramp driving 3 times? safer than humans(and many say that is already here), this would be a huge improvement for lots of people. Long commuters and overnight trips instead of flying, especially.

> for a moment take them at face value

Why would you do so?

This is like saying "Let's assume the sky is purple. Then, we can conclude that the sky is purple."

Can you elaborate a bit on how such testing is done, or share a good article on the topic? It sounds like a hard problem to need to get things right this bad, or else.

Not that long ago there was a few HN post about running on the metal that started off with this [0] (also called "Space Shuttle Style). Also speaking of NASA, they - and many other government departments - use a system called Technology Readiness Level (TRL)[1 -image] [2 - 1pg pdf]. This is used enough that you'll see it in HN comments. With humans on board, you are basically aiming for TRL 8. Look at the steps there and you'll quickly see that 4 years is pretty freaking fast. This not only includes code, but hardware. Everything has to be thoroughly vetted. In a typically contract you can go from TRL 1-3 in 6mo. 3-4 in 6mo-1yr. 4-5 in 1-2yrs. And so on. My guess is that the Tesla stuff is closer to TRL 5 or 6, since there is a driver involved. You'd need TRL 8 at level 4/5 to get fully autonomous driving approved. As it should.

As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying.

[0] https://github.com/kubernetes/kubernetes/blob/ec2e767e593953...

[1] https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg

[2] https://esto.nasa.gov/files/trl_definitions.pdf

> As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying.

I doubt more than 5% of developers have ever worked on anything beyond a TRL4. SREs, maybe,

Typically, everything in civil aviation is done to DO-178C. Almost everything we work on is level A, the highest level of criticality. Military stuff is generally much less rigorous, believe it or not.

The issue isn't the actual certification, it's the rapidly changing requirements and the standard for testing most software. There's plenty of libraries and tools I'd colloquially consider TRL 8 or 9 but there are tens, sometimes thousands, of people who can make a change to that software and push an update to millions of systems with little more than a glance from an SRE running `apt-get upgrade.` The nature of network facing software even requires you balance the need to keep your infrastructure stable with the need to keep it secure so you're always stuck in a loop between customers, support, management, developers, and operations where everything constantly changes.

Once software is operational, whether you are following DO-178x/33x, FDA's General Principles of Software Validation and other guidelines, ISO-whatever, or NASA/military TRL, there's a whole ton of stuff you have to do to make any changes once you reach a certain point in development. Since most businesses have the benefit of captive employees that can be woken up in the middle of hte night and terra-locked systems, they never even come close to reaching that point.

it's been a while, do-178 is concerned with software while do-254 is hardware (both for aviation, FAA standards). Depending on the criticality of the component (e.g. engine control vs radios vs entertainment system) they will abide by different levels (ie DO-178A ... DO-178E).

For both software and hardware there will be requirements for documentation, design, verification, and testing. Some even go as far as to requir the implementation of certain functional logic in multiple ways and having concensus logic (eg hardware logic to interpret GPS accuracy/confidence That ends up being broadcast externally).

There's also DO-160 that's concerned with environmental requirements (eg temperature, humidity, lighting).

This is what makes a radio that should cost $300, cost $5000, and why FAA certified aviation components are expensive compared to uncertified components (eg hobby airplane builds).

Parent comment really gives a great overview of the process.

One slight nitpick is that DO-178B or DO-178C are revisions of the standard and the associated criticality is associated with the Design Assurance Level (DAL) from A to E. One would say software is developed to DO-178C DAL B for example.

The DAL is determined before the software is started at the Systems level according to a process described in ARP4754A (Guidelines For Development Of Civil Aircraft and Systems) looking at the system architecture, hazards, and potential mitigations of those hazards, one of which is developing software to a certain DAL.

I agree, areas where reliability > power; outdated hardware is of less concern.

Same reason why New Horizon used MIPS R3000 CPU in 2006, where as it was also used in Playstation 1 in 1994.

Reply on news.ycombinator.com