Hacker News Reader

Comment by ericd | Hacker News Reader

Strong allegations, what is the death rate per million miles of FSD vs baseline? Is that a publicly known figure?

As FSD is a safety-critical product, the burden of proof is on Tesla to prove in a verifiable way to third partys that it is safe. Absent that, it is assumed to be unsafe. So, no, it is the weakest allegation possible and the default assumption.

As to the baseline rate, the baseline rate for police reported accidents in the US is 1 per ~500k miles and fatalitys is 1 per ~80M miles [1]. That is approximately 1 per 30 years of driving and 1 per 5,000 years of driving, respectively using the reported average of ~15,000 miles per year per driver.

As to the availability of FSD data, you can see my other comment:

https://news.ycombinator.com/item?id=36181877

[1] https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf

I still think that saying "Tesla chose to kill people by shipping a low quality system." is a strong allegation. I don't think it's reasonable to speak with that kind of certainty about an unknown. If your other comment is correct, that's a shame that they haven't been required to make that data public.

A rupture: https://pbs.twimg.com/media/EksIy3aW0AEIsK-?format=jpg

By your standards you can never deploy any FSD system anywhere, because such a system cannot be proven safe without deploying it.

You could do extensive closed-course testing with realistic conditions and then pay to have human drivers watching & ready to take over at any time. The problem is that this costs money and takes time, and Tesla was hoping not to do either.

Closed course testing to compare with humans would require about 800,000,000 miles of testing (10x length of average travel between fatalities). And that would just be for one version, every update would require that, and updates are about 100x internal version changes per year depending on how you count additional ML annealing.

That's about 80B test miles per year at about a capital cost of $0.50 per mile.

For a company that, say, spends ~10% on R&D, it would require the company be about a trillion dollars, which only Apple, etc. have reached.

Every other system has rigorous testing with trained safety drivers, planning interventions and working in short enough shifts to remain attentive.

Tesla does some of this, but then releases fast to the public... So far it has had some concrete consequences when they pushed this testing responsibility onto unaware people.

By that standard, every single existing transportation option is unsafe. We are back to hunting and gathering while living in trees, but let me tell you, even tree climbing can be dangerous.

They're talking about safety certification... And yeah, other transport components are safety certified. Autonomous software certification is a new enough area that the process is still being developed, but the EU has already instituted rules that specifically call out Tesla's actions as bad examples.

That wasn't the standard presented in parent comment:

> As FSD is a safety-critical product, the burden of proof is on Tesla to prove in a verifiable way to third partys that it is safe. Absent that, it is assumed to be unsafe. So, no, it is the weakest allegation possible and the default assumption.

And what you are talking about in this comment is quite different. Please don't move the goal post (or actually, sure, safety certification makes much more sense than verifiable safety).

I'd call that quote a perfectly adequate summary of certification. Can you rephrase what point you're making to the GP more clearly? It's not obvious to me how certification wouldn't be an answer here?

It is in the quote. Certified safe (this car isn't completely safe since no car is, but it meets our metrics so we certify it) is different from third-party verifiable safe (we have a proof of safety!).

This is such a inane semantic quibble. I was saying that Tesla must be able to prove (i.e. in the legal sense of meeting a standard of evidence) to a third party (i.e. a unaffiliated entity who does not have a conflict of interest, probably a regulatory agency or certification body), in a way that is verifiable by the third party (i.e. they do not require information from Tesla or they do use information from Tesla that can be reasonably believed to be accurate despite the conflict of interest), that it is safe (i.e. it meets a standard of safety generally agreed upon as acceptable for the problem domain). So yes, I was talking about safety certification.

Then go and just say that and don't go and use language that is much more strict in our own field (assuming HN average readers). As you say, you could have just rewritten your comment as:

"It is up to Tesla to get safety certification (that doesn't exist yet)."

That is obviously nowhere near what you wrote:

> As FSD is a safety-critical product, the burden of proof is on Tesla to prove in a verifiable way to third partys that it is safe. Absent that, it is assumed to be unsafe.

How does that in any way relate to safety certification?

I did not do so because I did not think anybody would misinterpret my statement as you did. I instead expected them to interpret it as the first person who replied to you did who found it obvious what I was talking about. It is in fact shocking to me that literally anybody would assume that I was declaring that they must prove that it is absolutely perfectly safe in a mathematically provable manner as that would mean that the reader thought I was such a blithering idiot that I can not even propose a minimally coherent argument.

I used extra words because I was expressing the core principle of certification which is to demonstrate to a third party that you achieve some set of agreed upon requirements. I did this because there is no current specific safety certification for autonomous vehicles, so I must appeal to the general principle underlying all safety certifications which is that you must demonstrate "safety" whatever that happens to be and that a product is not assumed "safe" by default.

Not to take away from the rest of it, but there is a very new process for L2-3 certification in Europe. As far as I know, only Mercedes has completed it and released a product (with TUV Rheinland), but TUV Sud is also offering homologation services for European type certification. The L4+ process for Europe is still in working group, with standards expected to be released next year (and 2025 for the UK).

Manufacturers are all pursuing self-certification in the US because regulators have been unwilling to answer petitions either way or commit to firm guidelines.

Again, certification (a matter of judgement) is a much lower standard than verification (a matter of truth). There is a lot of literature on the difference between the two words, and I assumed you chose to use the word verification intentionally. If that wasn’t the case, then it is better to choose certification when you mean that rather than verification in the future.

I think you're getting hung up on semantics. It's perfectly fine to call certified products "safe" in a casual sense, and the process of certification is proving that status to a third party.

Again, I wouldn't find these objectionable statements in casual conversation, even as someone who participates in the mess that is certification.

Note that any baseline should be on the same type of road e.g. FSD is probably already pretty safe on most highways whereas it's probably a coffin with wheels on a British country lane.

Reply on news.ycombinator.com