Hacker News Reader

Comment by ucaetano | Hacker News Reader

> As a point of reference - Google’s TPU V1, which is the one that Google uses to actually run neural networks (the other versions are optimized for training) is very similar to the specs I’ve outlined above.

The TPU V1 is a 2015 chip (publicly announced in 2016, and that Google had been using internally for over a year). 4 years is a big lag in terms of technology.

Additionally, the comment about "actually run neural networks " seems to be, AFAIK, plainly wrong. The first version was limited to inference because it could only perform integer operations, while V2 and onward support both training and inference, as they can perform float operations.

TPU V1 was never available to the public, but both V2 and V3 are available on Google Cloud. I don't have any info on it, but at this point, I'd expect all V1 chips to have been deprecated and likely removed from production, given the savings in space and power from V2 and V3.

View on HN

4 years is a big lag if you're not doing safety critical work. If you are doing safety critical, 4 years isn't particularly fast, but it's not unreasonable. If you've never worked in aerospace or automotive, you would be shocked at the requirements for software and hardware that can kill people if it breaks (assuming that Tesla is following the automotive equivalent of DO-178/DO-254, ISO26262)... I guess the automotive guys do have an advantage because they're not required to have independence for the various stages of verification, but it's still a big chunk of work (more than the actual development).

> If you are doing safety critical

Well, Tesla clearly isn't doing safety critical work on autonomous systems (it wouldn't have even launched "autopilot" if it was), so 4 years is indeed a big lag.

Tesla claims [1] that autopilot driving is safer than regular driving in terms of risk of accident per mile. Not sure if I agree with the way they've constructed their statistics, but for a moment take them at face value.

Isn't then the safer thing to release and encourage autopilot to the largest extent possible?

[1] https://www.tesla.com/blog/q3-2018-vehicle-safety-report?red...

That statistic is wildly and purposefully misleading. It compares the safety of Autopilot (which only drives on the freeway) with human drivers on any road. Accidents-per-mile are already much lower on the freeway.

Plus, the statistic ignores the fact that responsibility is actually shared between humans and the car. If autopilot gets into an unsafe situation and gives control to the human at the last moment, the car company might claim that the accident happened under human control. We have a long way to go before we can truly understand the safety of semi-autonomous vehicles.

>> If autopilot gets into an unsafe situation and gives control to the human at the last moment

We have lots of experience with the reverse, where the human drives but the autopilot rips control away when things go wrong. Traction control, radar-triggered braking, even basic ABS is a machine taking over from the human once things get hairy. Several AirForce fighters have systems that will sometimes ignore the pilot's inputs and save the aircraft from a crash. That's the real path for semi-autonomous cars imho: you do all the driving until the robot steps in to save your ass.

(I cannot wait for the day someone tries to sell a car that will automatically obey speed limits regardless of driver input. That's a stock to short.)

I'd even expect their human driven statistics not actually representative or useful for comparisons. Their cars are expensive, mostly excluding young and thus often unexperienced drivers. They also don't particularly target older people, the other group with high accident rates.

Controlling for all those variables is hard and not to their benefit, thus I'd never expect their pr team to do so. But I wonder if that might even creep into their automated driving stats... e.g. by drivers taking control in dangerous situations.

> for a moment take them at face value

Why would you do so?

This is like saying "Let's assume the sky is purple. Then, we can conclude that the sky is purple."

Well, it’s not autopilot, it’s guided steering and gas, which on a freeway is the “easy” autopilot path: trivially (for a human) identifiable routes, signage, infrequent low visibility occluded by tight turns, etc. I’d guess a) if you removed the driver from behind the wheel the accident per mile would skyrocket and b) humans already have a much improved accident rate on the freeway compared to general purpose driving. Is it better than a driver driving alone? Maybe—time will tell as more dose off behind wheel, although I am hopeful—but it’s hardly designed to operate without failure and should not be referred to as autopilot at all. It would help if Tesla provided a meaningful comparison instead of the disingenuous one they trot around.

Safety improvements on divided highways I think are to the point that driving is so boring for humans that any increase in safty ends up with offsetting human behaviors like, eating, texting, phone calls, looking at the scenery, falling asleep. If auto-cars can do on-ramp to off-ramp driving 3 times? safer than humans(and many say that is already here), this would be a huge improvement for lots of people. Long commuters and overnight trips instead of flying, especially.

Even if it's not now, their intention is to provide higher levels of SAE automation on the same software as time progresses, which means that the current hardware probably has to hit a higher design assurance level than the current software.

Sure, but intentions are irrelevant. What matters is what happens.

Can you elaborate a bit on how such testing is done, or share a good article on the topic? It sounds like a hard problem to need to get things right this bad, or else.

it's been a while, do-178 is concerned with software while do-254 is hardware (both for aviation, FAA standards). Depending on the criticality of the component (e.g. engine control vs radios vs entertainment system) they will abide by different levels (ie DO-178A ... DO-178E).

For both software and hardware there will be requirements for documentation, design, verification, and testing. Some even go as far as to requir the implementation of certain functional logic in multiple ways and having concensus logic (eg hardware logic to interpret GPS accuracy/confidence That ends up being broadcast externally).

There's also DO-160 that's concerned with environmental requirements (eg temperature, humidity, lighting).

This is what makes a radio that should cost $300, cost $5000, and why FAA certified aviation components are expensive compared to uncertified components (eg hobby airplane builds).

Parent comment really gives a great overview of the process.

One slight nitpick is that DO-178B or DO-178C are revisions of the standard and the associated criticality is associated with the Design Assurance Level (DAL) from A to E. One would say software is developed to DO-178C DAL B for example.

The DAL is determined before the software is started at the Systems level according to a process described in ARP4754A (Guidelines For Development Of Civil Aircraft and Systems) looking at the system architecture, hazards, and potential mitigations of those hazards, one of which is developing software to a certain DAL.

Not that long ago there was a few HN post about running on the metal that started off with this [0] (also called "Space Shuttle Style). Also speaking of NASA, they - and many other government departments - use a system called Technology Readiness Level (TRL)[1 -image] [2 - 1pg pdf]. This is used enough that you'll see it in HN comments. With humans on board, you are basically aiming for TRL 8. Look at the steps there and you'll quickly see that 4 years is pretty freaking fast. This not only includes code, but hardware. Everything has to be thoroughly vetted. In a typically contract you can go from TRL 1-3 in 6mo. 3-4 in 6mo-1yr. 4-5 in 1-2yrs. And so on. My guess is that the Tesla stuff is closer to TRL 5 or 6, since there is a driver involved. You'd need TRL 8 at level 4/5 to get fully autonomous driving approved. As it should.

As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying.

[0] https://github.com/kubernetes/kubernetes/blob/ec2e767e593953...

[1] https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg

[2] https://esto.nasa.gov/files/trl_definitions.pdf

> As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying.

I doubt more than 5% of developers have ever worked on anything beyond a TRL4. SREs, maybe,

Typically, everything in civil aviation is done to DO-178C. Almost everything we work on is level A, the highest level of criticality. Military stuff is generally much less rigorous, believe it or not.

The issue isn't the actual certification, it's the rapidly changing requirements and the standard for testing most software. There's plenty of libraries and tools I'd colloquially consider TRL 8 or 9 but there are tens, sometimes thousands, of people who can make a change to that software and push an update to millions of systems with little more than a glance from an SRE running `apt-get upgrade.` The nature of network facing software even requires you balance the need to keep your infrastructure stable with the need to keep it secure so you're always stuck in a loop between customers, support, management, developers, and operations where everything constantly changes.

Once software is operational, whether you are following DO-178x/33x, FDA's General Principles of Software Validation and other guidelines, ISO-whatever, or NASA/military TRL, there's a whole ton of stuff you have to do to make any changes once you reach a certain point in development. Since most businesses have the benefit of captive employees that can be woken up in the middle of hte night and terra-locked systems, they never even come close to reaching that point.

Everything we know about how Tesla operates as a company indicates that they do not freeze development of a technology or component and then spend four years testing and verifying it before deployment.

That's not exactly how it works. Development doesn't freeze - change requests, PRs, etc. can all be rolled in right up to the end and even after release if the appropriate process is followed, and in software, even in a safety critical world, you can pipeline updates out in a reasonably fast manner (we can get it down to about two weeks for minor updates). Hardware is an entirely different beast. You have to do FPGA development and prove-out, gate-level sims, timing accurate sims, thousands and thousands and thousands of simulated testbenches (hardware sim is orders of magnitude slower than software sim - hours per second, even on the most advanced equipment). Therefore, each testbench must be developed to test a very specific thing - in general, you can't just run your high-level software unit tests. Anyway, even if Tesla is the kind of company to roll software (doable) or boards (doable, but harder) right up to the last minute ... they do not (and neither does anybody else) spin a chip over and over again during the course of the development cycle. Speaking as the lead of a SW team that proved out a safety-critical custom SOC, it is colossally expensive. Every single iteration takes many months and millions of dollars.

You're agreeing with me. Tesla in all likelihood does not freeze development of a chip model for 4 years before releasing it. So you cannot compare Tesla's current chip with a Google chip that is 4 years old.

please read to the end nobody iterates chips right up to release. That's a software thing.

I agree, areas where reliability > power; outdated hardware is of less concern.

Same reason why New Horizon used MIPS R3000 CPU in 2006, where as it was also used in Playstation 1 in 1994.

But you don't need to train in a car, so you want to optimize cost/power for pure inference. Similarly if all you want to do is cheap and efficient inference, it isn't clear to me that TPU V1 are obsolete today.

All Teslas need to do is run inference. I'm fairly certain training is all done either in the cloud or on datacenter GPUs.

It would be very interesting if Google was selling Tesla EOL TPU v1 units. The specs are very good for what Tesla is trying to do.

4 years is not a long time considering this is a proprietary technology that even GCP customers haven't had access to for very long. It will likely be a few more years before any TPU-like technology is available in the mass market.

Merely being able to do math in floating point doesn't make a chip appropriate for NN training.

Indeed, but it seems that the V2 and V3 TPUs are quite appropriate for NN training.

there's no training going on in this case

you don't need to train in the car or to do massive scale inference, so you likely don't need anything close to the performance of recent TPUs

Reply on news.ycombinator.com