The End of Starsky Robotics
medium.com
medium.com
I worked at Starsky Robotics as a perception team intern after graduating high school. I will always be grateful for the team for the opportunity, it was a fantastic first job and everyone who worked there was very kind (especially Stefan).
Unfortunately, Starsky had effectively had no machine learning in 2017 (when I worked there), using solely classical computer vision techniques. This didn't match the company's ambitions of not using LIDAR and there was a strong stigma against switching to a deep learning approach. At the time, very few object detection models had public implementations and I spent a lot of time trying to get a YOLO9000 and RetinaNet implementations running at real-time speeds. Frustrating, as a small startup the labeling services kept screwing us over by returning poorly annotated images.
I think what I took away from the experience is that deep learning in domains with long tails requires a enormous investment in a labeling pipeline - dwarfing the computational aspect - to get decent results. I don't think any solutions are on the horizon that will allow us to bypass this reality. You don't see improvements between Comma.ai and Tesla because it's about the improvements far out on the tail.
YOLO9000: Better, Faster, Stronger - https://arxiv.org/abs/1612.08242 (2016)
Data is the problem. Everyone has the algorithm but not enough people have data (especially labeled ones)
Detectron was open sourced in 2018. R-CNN didn't have any public implementations (there was later a Keras implementation that didn't get the same performance as the paper reported). TensorFlow models added some object detection models a few days after I started my internship, but had various issues at the time. SSD and YOLO both had public implementations, YOLO's being in it's own C based framework.
It's a completely different landscape three years later.
Not to mention that nvidia has thrown numerous open source efforts over the years. If SR was under the impression that 2017 was a dry year for open source deep learning vision systems - I can understand why it didn’t do very well technology wise.
Disclaimer: have been doing deep learning open source and research over the years. Have touched all major frameworks in the market.
I wonder when the market will start listening to people who actually WORK on the problem.
> which means that no one should be betting a business on safe AI decision makers. The current companies who are will continue to drain momentum over the next two years
Sounds about right.
-----
Here's me over 2 years ago simply quoting people who worked close to the problem:
https://news.ycombinator.com/item?id=16353541
Here's my negative scenario: self-driving is "AI-complete"; you can't really hit all the edge cases without solving AI in general, which is more than 30 years away (Kurzweil is the wild optimist and predicts 2045).
You CAN use self-driving in limited circumstances, but those limitations are precisely the ones that make driving yourself around more attractive. The expense doesn't go down as quickly as anticipated because of this. They are a niche technology for DECADES.
This looks about right, and I'm not claiming to be prescient, just basically saying what Chris Urmson and Bill Gurley already said years ago. It's weird to me that there's still so much money chasing this pipe dream.
It was a sign when Google spun out Waymo. If they really believed in the product, it would be called something like "Google Self-Driving Cars", not "Waymo".
----
edit: it's easy to be negative, so on the positive (and contrarian) side, I believe rideshare is undervalued, and has a long way to go. I bought both Uber and Lyft stock at $16 yesterday (they may go down again, but ping me in 2 years). This comment in this thread is a nice take and makes me even more optimistic:
This also completely ignores the halo effects of these systems that vastly increase the economic output of the region. This is why it is often logical for the state to pay for the system, since the costs are centralized and the profits are distributed.
https://nymag.com/intelligencer/2017/06/lyft-reinvents-the-b...
The point is that we already have self-driving vehicles. You press a button on your phone and somebody drives you around. Whether it's a premium ride, a shared ride, or a bus is an orthogonal issue to whether the driver is human, which is a matter of cost.
The fact that it's a cost issue makes it look even worse and further out for driverless.
If there is demand for a shuttle along predetermined routes, it would already exist with a driver, and that's what Lyft Shuttle is if I understand correctly. I haven't seen it in SF or heard of anyone using it. It's definitely possible that they could figure out the product and it will become popular, but that has little to do with self-driving.
I see it the other way, they slot immediately and easily into already existing businesses and if the only difference is cost consumers are likely to swap over.
I'm curious what specific statements by Chris Urmson you are referring to. When he was in charge at Waymo in 2015, they announced that their solution is good enough to drive a blind man around town. And this was before they acknowledged they're using teleop.
The Urmson one is from March 18, 2016 talking about a 5-30 year time frame, and the Gurley one is from April 2017 talking about a 25 year time frame.
Without any more context, "drive a blind man around town" has about 3 decades of wiggle room in it... If you look at Waymo's recent blog posts, they have similar words that sound good but leave wiggle room, and they've been doing that forever.
Over the next five years, hundreds of companies will claim to have successful self-driving cars, Gurley said. But he said he’s reading between the lines of press releases, which often tout tests in very controlled environments that do not totally reflect the real world.
“The part we haven’t figured out yet, the last 3 percent, which is snow, rain, all the really, really hard stuff — it really is hard,” Gurley said. “They have done all the easy stuff.”
-----
Urmsson was trying to having it both ways, but if you read between the lines it's obvious what he was saying:
Urmson put it this way in his speech. "How quickly can we get this into people's hands? If you read the papers, you see maybe it's three years, maybe it's thirty years. And I am here to tell you that honestly, it's a bit of both."
He went on to say, "this technology is almost certainly going to come out incrementally. We imagine we are going to find places where the weather is good, where the roads are easy to drive — the technology might come there first. And then once we have confidence with that, we will move to more challenging locations."
In an interview, a Google spokesman agreed that Urmson was describing some aspects of the project differently than the company had in the past. "Yes, there was some new stuff in there," the spokesman said. "That was obviously the intention of the speech: To say some new things."
-----
That was OVER FOUR years ago. People didn't get the message and kept pouring money into it. Prediction: if we get economically viable level 4+ self driving in the next 10 years on a $1B+ dollar scale, it will come by way of a breakthrough, not by an extension of existing tech. That is, by throwing out tens of millions of lines of code that's been written and billions of dollars in hardware that's been designed and manufactured, and taking a different approach.
If you are correct about that, isn’t it strange that we would give birth to a intelligence and immediately assign it the job of Taxi Driver? Surely it would be an ethical minefield?
This is exactly the problem with data hungry machine learning approaches, specifically deep learning (and that’s without even mentioning the compute resources necessary to learn). The only way to circumvent that is plausibly apply better inductive biases, and fundamentally rethink what the field considers important.
Thinking of computers as getting more "intelligent" like humans is a category error -- computers are dumb matter configured in an intelligent way by actual intelligence (humans) to preform certain tasks for us. We get better at telling them how to preform certain tasks (software), but there's no reason to think we're moving along some continuum of intelligence towards us.
That's the premise of deep learning - inducing high level concepts from experience, without manual feature engineering.
DL models are good at induction, what they can't handle is generalisation (being accurate outside of distribution). And self driving has a long tail, that's why it's so hard.
It may never be enough to beat the best human drivers, but it only needs to beat most human drivers to be worth it. We're not that far off.
Your objection reminds me of the skeptics of spell check and grammar check. In principle, a perfect spell and grammar checker would also need general AI to fully understand a language and what you're intending to express. Fortunately, imperfect spell and grammar checkers are all that most of us need.
Most of the engineering team has spread throughout the AV industry, with most folks going to our neighbors and fellow YC company Cruise Automation. Some are at Waymo, Zoox, AutoX, and some purposefully exited the AV space entirely.
I joined Applied Intuition to help build out Simulation and Infra for other companies producing AVs/robotics.
There are a few folks I know of who are still looking for their next roles in the BizOps/PplOps side, which has been especially hard during COVID-19 season if anyone wants to do some linkedin stalking.
Nevertheless, we found an incredible amount of industry and investor resistance to our teleop-dependent approach.
Why was this the case? I can't see the harm in using teleop.
There are still a number of folks who think that teleop can never be safe - surprisingly sophisticated people hold onto that dogma. We wrote a whitepaper about it for investors, and I'll share that at some point as I open up more and more of the Starsky vault.
All parts of an autonomous system break. Safety engineering is measuring those breakages, and designing a system that is safe when it inevitably breaks.
If you're the operator, you can choose to only drive on routes that should have sufficient connectivity. If your remote driver is only issuing high level commands (i.e. not responsible for safety) latency starts to stop mattering so much.
The problem here is the availability bias - you've seen your phone fail so you know that telecom links can fail. As a layperson you might not think about how the rest of the system suffers from the same limitations - but they do. You engineer around them.
So it is more than sufficient if you can show that AVs - on average - cause less harm than human-operated vehicles.
To your specific question: when the AV loses connection, it would do the same as a human driver when a tire is punctured: turn on warning lights, slow down, and stop at the roadside. Like in other car failure situations, that might cause an accident in some cases. However, that is fine, as long as it is rare enough.
(disclaimer: I'm not working in AV tech, I don't know if current AV technology handles this case as imagined)
1. Referencing page 5 of your VSSA (1), how were you able to quantify "When weather, traffic or other conditions made driving unsafe." It seems like a tricky problem because this region of the ODD is not discrete such as {warehouse, ramp, highway} but rather depends on the quality of your cv stack as well?
2. In hindsight and with the company behind you, do you think it make sense for each car companies to do self reporting? Or should there be some sort of government oversight/technical validation process done by external 3rd party?
3. I would love to chat more! It must be a very emotional time for you, so I would understand if you do not wish to speak about it. But if you do, could we set up a time at lingxiao@seas.upenn.edu?
[1] https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1...
3) Sure! It isn't too hard to figure out how to email me, so send me a note or DM on twitter or LinkedIn or whatever.
2) AV has been a great testcase for federalism, and I think we've seen different regimes work differently. For the stage that industry is at, I think that insurance requirements is sufficient (insurance carriers have the sophistication to figure out if you're being unsafe). California's regime is overly-prescriptive in such a way that it confuses things more than it improves safety.
To be clear - if we were closer to deployment I'd advocate for a way tighter regulatory regime.
1) We figured out a programmatic way to measure ODD. Trying to specify all versions of ODD is a fools errand, and you're right that it would change a lot by how good you stack is.
ODD = Operational Design Domain. The conditions your system can drive safely in - rain, snow, sunshine, glare, high traffic, no traffic, pedestrians, etc ad infinitum
Sounds like most of the engineering team does not share the founder's perspective on the AV industry.
Best of luck, and I'm looking forward to your next venture!
> It didn’t matter that that jump from “sometimes working” to statistically reliable was 10–1000x more work.
There's 2 states of functionality:
1) It doesn't work
2) It sometimes works
The inverse, for disk drives: Failing and Failed.
Think about apps/services. You could say that your app is working, but over a long enough time period, it is only sometimes working. It's working while you have disk space, free memory, and a working network connection. It's working while your business assumptions hold true. It's working while your datacenter has power. We've developed strategies for managing all of these things; for load balancing and Active/Active hosting. But even with that, it only sometimes works.
---
With all that, I think it may be useful to think of self driving in terms of tasks.
If you can put a box around what you expect a computer to be able to do, you can define tasks that will always return a reasonable output.
The more tasks a computer handles, the easier it is for the human in control (think driver assist, like lane keeping and automatic cruise control).
If you add enough tasks, and perform them well enough, maybe you can take the human in control out of the vehicle.
I think I'm in agreement with the authors that I don't see the day when there isn't a human in control. Or the other way to say that is that if there isn't a human in control, sometimes your AV will just stop.
To the author, it’s interesting to read how quick you rose and fell, but it’s good you got out now instead of holding on for years and years.
What do you think is next?
I’m honestly tired of the lack of progress in self driving, and was wondering what anyone’s thoughts were on home robotics. I feel like that’s the next big thing that VC’s will pour billions into.
[0]https://media.gm.com/media/us/en/gm/news.detail.html/content...
I am sorry you could not get investors to believe more in what you and your team, especially as you required a lot less funds than many other companies in this arena. I also thought you had a clear business case (I worked in ride-hailing and also logistics so understand some of the problems in this space).
I wanted to ask you a question: I am building a startup in the dash cam video analysis space. We are building a large and geographically diverse dataset of road videos, where our users can annotate/label the data. We then are going to look at detecting specific events like accidents and edge cases on videos. Do you feel this type of business, the data we collect, and insights we generate would have value for a AV startup?
All the best in your next move. Stay strong - you can be proud of what you and your team did.
We had really strict rules on what we wanted our data to look like, and were very specific about subject matter. We probably wouldn't have been able to be a customer.
Our intention to deal with accidents/edge cases was that if anything looked outside of ODD (as in, not perfect driving conditions) we'd execute a MRC (pull to side of the road, or stop at next exit). Relatively easy way to solve most of the really hard edge cases.
Well done.
I'm going to assume the founder of a self-driving truck company knows what he's talking about.
But at the same time, I have a hard time reconciling that with the fact that I sat in a car that drove itself all around San Francisco, dealing with a ton of edge cases.
Maybe we won't get to a 100% drive-anywhere-you-want-car in 10 years, but to be fair, a lot of humans aren't capable of driving a car anywhere either.
There are a lot of LA drivers who can't drive in snow, for example. I was one of them, until I got practice, and even then, I'm not that safe at it.
I think as long as we set the bar at "drive anywhere a car can go with 100% safety" we will never reach that bar.
But if the bar is at "drive as well as a human in most of the places humans drive", well, I've already seen a car that can do that.
If it was super mapped and following a fixed route with object avoidance; the difference between a car that can go on an HD-map track and one that can go point-to-point in a city is also maybe a decade.
We humans have limited empathy for what is really really really hard for computers. Many know that's true for an app, but somehow ignore that knowledge when it comes to autonomy.
I really resonate with the challenge they had making safety "sexy" from a coverage point of view. It seems it shares that attribute with "security" which, when done well, means nothing happens.
I had a lot of hope that we would get four lane (two each way) "freight ways"[1]. These would be paved roads designed specifically for autonomous trucks to move freight from one point to another. With sufficient freight ways between key cities on the coast and further inland, and efficient way to move goods with somewhat worse costs than rail and quite a bit better in terms of reaction time.
[1] Like a free way but designed specifically to exclusively service autonomous trucks carrying standard containers.
To paraphrase Churchill, trucking is the worst form of transportation, besides all the others. If you can put freight on a train, you do. It's slower, point-to-point, and less good a less-than-trainload freight than trucks are.
Freeway costs roughly $4m/lane/mile. Building a duplicate system for trucking would make UBI look cheap. Truck taxes also pay for much of current hway infrastructure, so you'd need to replace that $
Will investors lose their loss memory and a whole new set will invest in the space in say 3 years.
Full autonomy isn't necessary. And I don't know if it's even profitable for trucks.
My bet is that nearly all of them get COVID, with a higher hospitalization / mortality rate. Could knock 30% of capacity off the road in the next 2 quarters, which would mean food shortages.
lol.
I used to live on one of the side streets in MV that Google's AV cars trained on. Saw three in a row once. They were usually the only traffic, so that annual report of millions of miles travelled was meaningless - they might as well have driven arounf the Safeway parking lot at 5 am.
I guess passenger-carrying quadcopters is next. Oh wait ...
A better solution for robotaxis - just avoid the weird intersections.
Just curious, why would there not be a business model for robotaxis?
Driverless: the company needs an operation, real estate, and staff comparable to a big auto rental company to do all that. Plus the engineering and technical staff required for autonomy.
Even with startups doing autonomous shuttle buses, which works at low speed, nobody is making money in that space. It's all demos.
I recently listened to a podcast episode of the Autonocast[1], where they interviewed a Harvard Researcher who claimed the economics of Robotaxis just don't work. Very interesting listen.
[1] http://www.autonocast.com/blog/2020/3/11/177-ashley-nunes-on...
Early robotaxis will have tight geofences. Makes them particularly bad competitors to poorly paid people who are willing to drive wherever.
One additional factor seems to be that Uber drivers tend to underestimate the depreciation of their cars, so they effectively subsidize what otherwise would be Uber's capital cost.
And THAT's something you can't get a robot to do.
High level - Uber and Lyft have done a terrific job getting people to work for less than minimum wage. I've met quite a number of uberdrivers who work in Robotaxi-Ready cities (think Chandler, Tampa, etc) who work 12-16 hours/day, 6-7 days a week, and make $20k/yr after depreciation.
While sensors are getting cheaper, most teams respond by throwing more sensors on the vehicle. It seems unlikely there will be a <$20k robotaxi anytime soon.
TL;DR: Humans are cheaper than robots.
AKA - if people are willing to do it for cheap, it probably won't make sense to get a robot involved.
Probably won't help with any of those folks opposed to UBI because "people get meaning from work." Proponents of that view must know a hidden trove of self-actualized retail workers.
This is the key point. The new DNN approaches can outperform the classical techniques, but only when they can exploit vast amounts of training data. The dramatic successes of Deep Learning all depend on either unsupervised learning against enormous raw datasets (BERT, GPT-2, word2vec, etc) or games, where you can generate unlimited quantities of labelled data by playing the game against your own agent (AlphaGo, AlphaStar, OpenAI Five, etc).
I'm sorry things didn't end up in the exact way your team may have hoped. I hope you can take pride in everything you accomplished. I wish you all the best!
[1] https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1...
The business case always seemed clear to me and now reading this, I wonder if there is a case to be made for an engineer and a trucking operations veteran to build a business that requires minimal capital (or maybe even is bootstrapped?!), to take it across the finish line?
It's not for the faint of heart - truck drivers and engineers follow a very different set of social norms. It's hard to be the hardass boss drivers don't want to fuck with while being the cuddly CEO engineers like the culture of. That might be a future blogpost.
The point about safety being a low priority with investors makes sense but is unfortunate. I hoped that Google's investment in Waymo would push past this hurdle, but they're latest funding round makes it clear that they also have to deal with investor wants over the success of the technology. Really hoping to see someone get the funding to make this tech work reliably in the future and apply the same safety-first design that Starsky used.
The issue is that if I paid a ton of money for something, I do generally expect to get what I paid for. And in this case that's not gonna happen.
[*] Hypothetically, I'm not presently a Tesla customer.
I've had 30 hours of driving in a Tesla, and I had to intervene 6 times, twice with no warning. That doesn't mean that Autopilot isn't a useful tool, but it does require operator diligence.
Umm, what? Source?
My quote was from a person internal to Uber. Below is a TC article citing a public report on US Data. Numbers in that are roughly 50/year, or a fatality in America every week. Every day may have been a stretch, but I'd guess you could deduce that Uber drivers worldwide are involved in a fatality accident at least every day.
*also - this is obviously a lower rate than non-Uber drivers per mile
https://techcrunch.com/2019/12/05/ubers-fatal-accident-tally...
Humans do it really well with just two cameras. It's not a hardware problem; it is entirely software. Whether self driving is possible or not with current AI techniques is debatable, but we're not waiting on any advances in hardware to do it.
If you don't mind, I think I'm going to steal that quote. It makes a really good point very succinctly.
It's a pithy response that undermines the challenge of a problem nobody has been able to solve even with years of effort and billions of dollars.
As the OP said, "it's not a hardware problem," in that the quality or number of cameras, sensors, etc. isn't the bottleneck to solving this problem.
That level of resolution doesn't matter at all for driving. People can drive just as well through a video feed, like Starsky was doing. Yes general AI does not exist yet, but my point is simply that the parent made a comment about the need for hardware which is simply not true.
I can tell you from experience this is false. It usually works, but when it doesn't you're fucked. People wildly underestimate, by orders of magnitude, how many and how complicated the edge-cases are for self-driving.
Has Elon musk invented artificial general intelligence? If not, the point isn't a good one at all.
This is because we're totally unable to come up with a coherent model for why the emergent behavior occurs given the input data and NN training. We know how individual elements of the NN work, and we can describe how the training system works. But the whole notion of repeatedly letting perturbations in a control value or control values dictate the entirety of the performance of a system is nuts.
The only way to determine how a NN will perform in a given situation outside of the training set is to actually feed it the stimulus and check the outputs. Given the amount of stimulus that we as drivers routinely get, it's impossible to say with any degree of certainty that a self-driving car that is built on an NN classifying engine will accurately classify everything in all situations and lighting conditions, because it's impossible to feed it a training set large enough to encompass those situations.
That leaves the question of making a classifier that is better than humans. And whether an NN is better than a human depends very much on the situation. We could make some statistical arguments, but when you're gambling with peoples' lives here it becomes difficult to tolerate such arguments. It's easy to be blase about it until it's your child chasing a ball in front of an AV, at which point any discussion of the statistics is academic.
What makes this challenging is that there are more such scenarios that might occur than there are grains of sand, or stars in the universe.
Those teams end up thinking they did all the necessary work and then go test, just to find out that there's a new edge case. At Starsky we called it Safety Theatre
FTFY. You can thank me later.
We don't fully know how the human hardware works either. Or rather, we don't fully know how the human hardware + software works.
I have no idea whether Tesla will or won’t succeed. But they do have one major advantage over just about every other AV company out there, which addresses the point above. That is, their huge network of camera-equipped cars (a million and counting) provides probably the deepest, richest AV learning dataset on the planet, and probably by orders of magnitude. If accessing the dataset and thus novel edge cases is one of the major challenges in AV development, Tesla is very well placed.
> ...modifying the roads specifically for such cars, and segregating them from human drivers.
You were #this close# to answering your own question! Answer: https://boringcompany.com
Additionally, with highway-like vehicle spacing and high capacity stretched Model Xes (as the Boring Company proposed), the throughput is the same order of magnitude as a typical subway line. With denser spacing, potentially even higher.
Anyway, the question is about what Elon's backup plan was. This is it, regardless of whether it works or not. (And there's plenty of reason to think it won't, to be fair!)
Level 0-5 is terminology used by SAE to describe the range of fully manual (Level 0) to fully autonomous (Level 5). One would expect that in an article about Autonomous Vehicles, L{0,1,2,3,4,5} would describe the level of autonomy that the vehicle provides, but the article instead uses these to describe arbitrary thresholds of a graph demonstrating the capability of an AI.
I'll update at some point with different notation.
Not sure why l meant anything in some other model would affect its reading. Good to me.
Things that help recovery from burnout: long walks, therapy, exercise, sleep, meditation, and spending time with people you love.
The public are only ever going to accept perfection which prevents us from getting our hoverboards here.
I think the plateau of productivity will be really awesome computerized copilots and safety assists/warnings. The human computer driving team could Be quite a combo of designed right.
Realistically, most of the AV talent will go to industries that can afford them. Salary isn't super elastic.
Excited to see what you do next.
Uber drove an autonomous truck on a live highway to deliver beer for Anheuser-Busch back in October 2016
https://fortune.com/2016/10/25/uber-anheuser-busch-first-aut...
To take the person out of the vehicle, the system itself needs to be able to self-monitor and pull itself over if it stops working. That's the big lift.
No idea what that advance may be but it’s out there I think
An example of an additional sensor would be an infared camera. I don't know how well infared cameras work during the day or when they are occluded, but they seem like they can provide an additional datapoint to classify an object as "person" for example. That way, it doesn't matter if they're wearing a paper bag Halloween costume, the car still knows what's underneath.
Physical modeling seems like it would help with prediction. If a deer is idle on the side of road ahead, a car with a physical "deer" model would account for the probability that the deer will suddenly dart across the road and the speed at which it can possibly do so. It will slow down and watch for certain behaviors, such as the deer standing up from a prone position, and modify its probabilities accordingly. That is, after all, what we humans do. The ML wouldn't need to perfectly model all animals. Some animals would be put into a "mammal" class or more vague "animal" class, paying special attention to size and stature.
Additionally, depending on how long it might take to develop truly autonomous self-driving cars, we might want to develop self-driving car friendly infrastructure. I'm talking about AV-only lanes with safeguards in place to minimize variability, vehicle-to-vehicle communication, infrastructure-to-vehicle communication, and better than average maintenance. It seems like it might be worth it, given the gains to be had.
Here's an underappreciated angle to the AV-friendly infrastructure: it's a hedge against incorrect predictions. What if truly autonomous vehicles are actually 15 years away? In that case, if an "AV heaven" is developed in a corridor somewhere in Los Angeles 5 years from now, that'll be the only place where the AV dream is realized for the ensuing 10 years. The profit from "AV heaven" can be reinvested into AV development. It can also be used to expand into other areas. Moreover, AV heaven might galvanize the government, public, academia, and investors to eliminate any bottlenecks in AV development. There's no immutable stone tablet, only viewable by God, that says "AV development must take X years." We can modify the timeline to some extent. Lastly, AV heaven would allow us to observe the true value of self driving cars. What if they're better than we ever imagined? What if they radically change human behavior? What if they increase development in surrounding areas by a lot? What if they change the nature of work and where people live by more than we even expected? What if self-driving trucks + last mile bot delivery radically push down prices on everything and enable previously impractical business models or behavior? We actually know many of the changes that AVs will bring, but, critically, we don't know the magnitude of those changes.