California suspends Cruise's autonomous vehicle deployment
reuters.com
reuters.com
Today’s suspensions are based on the following:
13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
Misrepresenting safety seems like a big deal.
[1] https://www.dmv.ca.gov/portal/news-and-media/dmv-statement-o...
Edit: This Vice article [2] says Cruise tried to withhold pedestrian injury footage from the DMV.
> In the Order of Suspension, the California DMV said that the Cruise vehicle initially came to a hard stop and ran over the pedestrian. After coming to a complete stop, it then attempted to do a “pullover maneuver while the pedestrian was underneath the vehicle.” The car crawled along at 7 mph for about 20 feet, then came to a final stop. The pedestrian remained under the car the whole time.
> The day after the incident, DMV representatives met with Cruise to “discuss the incident.” During that meeting, Cruise only showed footage up to the first complete stop, according to the Order of Suspension. No one at Cruise told the officers or showed any footage of the subsequent pullover maneuver and dragging. The DMV only learned of that from “another government agency.” When DMV asked for footage of that part of the incident, Cruise provided it.
[2] https://www.vice.com/en/article/4a3ba3/california-dmv-suspen...
They manage to avoid collisions by driving extremely conservatively, but the way they traverse, say, a left turn against traffic is absurd. They slow everyone down, including emergency vehicles and public transit, by performing far below the level of most Human drives.
They don't work in the rain, they can't handle construction, they block garages and driveways.
Waymo vehicles are objectively far better. They drive like Humans do. Still some issued with weather and construction, but they work well alongside busses, trucks, and private cars without slowing anyone down.
* Waymo made a right turn up a steep hill. Two lanes each way. It then pulled a bit into the left lane abruptly and I didn't get why until a split second later a skateboarder was crouched down and went by on my right against traffic. There was zero chance a human would pull that off. Not enough time for a head check or mirror check. There wouldn't have been an accident but the Waymo clearly has insane reaction time and vision advantages - and uses them.
If it was a human that person would have been dead, no question.
Sometimes I'm riding in a Waymo (which I do every single day, 4x) and it does something, and I double take. Then a second later I see whatever it was it reacted to, and I'm like "dang, why did I doubt you, robotic overload?"
The best part is that it works the same way in the day time vs the night time.
Magic.
This was the turn - https://www.google.com/maps/@37.7337193,-122.4350752,3a,75y,...
* Not recommended in SF. But I imagine on the 280 it would be great one day.
You say this but as a motorcyclist I've learned to have my eyes everywhere. I suspect many others would say the same. You acclimate to the vehicle you most often control: if that's some woolly handling sluggish SUV then I suspect, no, you probably wouldn't have done well in this situation, but if you were more used to driving a vehicle that requires more attention I suspect you'd do better.
Ultimately, I don't think the driving screw ups were that big of a deal (just my opinion). The big deal though is the moronic lack of customer support. How hard is it really to have a customer support staff a phone call away that can reassure the customer and either debug the issue to get them sorted out or at least offer alternatives. It would do so much to improve the users perception.
So, typical Google... Phenomenal engineering, and horrible customer service.
Maybe it was a just hit piece though and they cut out some contextual video? No idea.
_defensive driving has entered the chat_
Seriously, anyone here who rides motorcycles can pipe in, but some of us out there never assume anything on the road and thusly avoid a lot of trouble because of it. Assuming incorrectly is the root cause of probably 99.999% of accidents.
i'm sure this is sometimes true, but at least sometimes i'd guess that the route selections would make more sense if you saw the conditions and traffic on the route the car didn't take you on.
I hate that they use giant SUVs, really inappropriate car for small city streets even if people love SUVs.
Some over-cautious stuff like mentioned above but also some lane changes and adjustments that were better than what I would do.
I still prefer to drive myself so I’ll probably turn it back off at the end of the month, but I’ve got high hopes for when I’m older.
Waymo vehicles impress me, I've seen them perform skillfully like you describe.
Also, isn't there an implicit assumption that all Waymo "drivers" would perform in exactly the same way?
This isn’t to say the Waymo car did a bad job. It sounds like a good choice in the situation.
Wonder what it would have done if it didnt have room to pull into the left lane without hitting a car. Or perhaps another skateboarder.
Waymo has way more sensors. Most importantly, it has Lidar.
From my perspective as a person who lives in San Francisco and also bikes a LOT: human driver also do not perform acceptably.
I counted 14 cars without headlights on after sunset on a 2 mile bike ride yesterday.
I counted 5 cars parked in either a bike lane or active traffic.
2 cars took blind rights (no stops) onto Polk from various alleyways.
This was a good day, FWIW.
Cruises have obvious problems, but my hope is we don't use humans as a baseline, because they're not very good either.
EDIT: This comment is pretty controversial for whatever reason, but said differently - why would we allow autonomous cars to kill 30k Americans every year (the same allowance we give human drivers)?
That seems like a comically bad baseline.
I've observed all the things you did (in fact, all of them on Sunday). But I don't do any of them, and you probably wouldn't either if you ever used a car.
I'm comparing self-driving cars to an average Human, who truly does drive acceptably. Over 99% of people drive acceptably. The 2 dudes on I-5 without headlights at dusk are a problem, but there are Hundreds of vehicles in sight driving very safely.
> That seems like a comically bad baseline.
Now consider a version of autonomous cars that would kill 20k.
The argument to switch to those cars, to save 10k lives a year, has to use 30k as a baseline.
If we don't use 30k as a baseline, then we end up saying "20k is too many, stick with the status quo", and then more people die.
We should do our best to keep standards high and increase them as technology permits, but there is no reasonable way to avoid using the current numbers as a baseline.
This one could be fixed tomorrow by adopting regulations that other advanced countries have had for decades.
Not just the kind where they try to make a yellow that they clearly weren’t gonna make, but also they stop at a red light, wait to see if there’s cross traffic, and then intentionally run the red light.
Did the bike lanes have a "no parking in bike lane" sign? Because (and I learned this by getting the question wrong on my CA license test when moving here) it's not illegal to park in bike lanes in CA.
The bar for AI should be way higher than just "outperform mediocre humans".
Waymo rides feel much more predictable and human like.
Although note that “human like” and “smooth” rides do not necessarily mean safer, but qualitatively Waymo feels that way.
Cruise's failures, in my view, have come broadly at the long tail where their cars stall out entirely in the road due and it cascades to other Cruises.
I realize I don’t have a real concrete sense of just how many of these there are. Like is this a whole thing already?
Edit: thanks for the responses…. Good gravy…
Can't wait for them to be able to take freeways over to Oakland and down to Cupertino.
Are they performing worse, or are they just making a better risk tradeoff? Human drivers kill a lot of people every week.
Cruise cars do not perform acceptably in SF but they probably would if they deployed in the bazillion mid-sized cities out there with wide streets and ample shoulder space.
> driving extremely conservatively
> traverse, say, a left turn against traffic is absurd
> far below the level of most Human drives
[1] https://www.fastcompany.com/90702355/40-of-san-franciscos-tr...
[2] https://www.nhtsa.gov/press-releases/speed-campaign-speeding...
___________
> Waymo vehicles are objectively far better. They drive like Humans do. Still some issued with weather and construction, but they work well alongside busses, trucks, and private cars without slowing anyone down.
I'd take this further. Given that self-driving cars not a fashion/emotional purchase, we should similarly tech-them-out to improve collision outcomes. Make the car collide with a pedestrian at their feet rather than the torso. Make it impossible for someone to go 'under' the car. promote outside-airbag design to minimize collision impact.
For starters, I support mandating driverless cars for all people with DUIs (at any phase in life), the elderly and those who have been in serious at-fault-accidents. Yes, self-driving cars are worse than the average human driver. But these people are clearly in the bottom 10 percentile.
Waymo's are far, far better. I ride one every day, and trust them more than I trust 99% of Uber/Lyft drivers.
But now it seems their strategy is paying off, at least in terms of developing an acceptable solution. It still remains whether they can monetize it well, but other people will have to more or less adopt Waymo's model or they won't have a business.
Knowing about software, if the key differential factor is how well you can handle a long tail of edge cases, it'll take years before others can compete with today's Waymo in fully autonomous self-driving cars. If you don't have an architecture and culture to obsess about edge cases, it could even be easier to tear down the software and rebuild.
maybe it's more compelling to say: as an average driver, EVEN I can see that Cruise is far less competent than Waymo.
I spent a year in the bay area, working in the city for about half of it. It completely destroyed any joy I got from driving. I've been back in Ohio for 10+ years now, and I still hate driving.
It was that bad then, and I hear it's only gotten worse.
You've just described the average Subaru driver in the Seattle area. How is this notably different? Should they be banned from the roads as well?
And this brings a question to me: How did they create, develop, this post-collision routine? Afaik, the overall program/system guiding a self-driving is some combination of trained neural-network routines and more standard robotic feedback loops. But "what to do after a collision" seems like one of those corner cases that neither programming approach would or could deal with so what development created this routine? It would have to be pure ad-hoc-ery and that suggests that in Cruise's approach to the last-mile/endless-corner-cases problem of self-driving cars is just that, a series of ad-hoc fixed. And this approach can neither be safe nor can it scale.
And further, I have to presume Cruise and the others have a bay of human pilots that are called-on to deal with difficult situations. Why didn't the "you've just had a collision" signal not pull a human? (maybe it did and not saying that is their problem but otherwise..) The main explanation I can think of is ... the car's routines are already calling pilots a lot and the team is working feverishly to reduce this. An ad-hoc pull-over routine is intended to reduce those calls back to the pilots. Admittedly this is speculation but if true it reinforces my conclusion above.
And then there's a momentary thought of this, and then I think how bad/hard it would be. Sitting there in a "call center", "Ding!" and you're suddenly thrown in to a car mid-collision, needing to take instant surveying of the situation (like "there's someone partly trapped under the vehicle", "am I in the middle of an intersection/interstate", etc., et al.
Even getting past that I think such a job would play absolute hell on epi, norepi, dopamine, etc., response and those people would burn out, quickly (and I say that as someone who spends a lot of time waking up to alarm tones going off before jumping in an ambulance or fire engine...).
I’m just going to assume it’s a liability thing:
* Cruise gets in an accident and stays in the road, passenger gets out and gets hit. * Cruise gets in an accident and pulls to the curb, passenger can get out on the curb (and if they don’t, and get hit, Cruise can claim it pulled over).
I could see this decision being made (or reviewed by) legal. Which sounds like a truly awful way to build a product.
I agree with you on the adhockery as well. Sounds like it.
So the city and DMV focussing their efforts on this corner case driving systems seems like dereliction of duty on the City attorney's and DoT's part.
[1] https://sfstandard.com/2023/07/01/ask-the-standard-san-franc...
> Firefighters obstructed the sensors of the driverless car to alert the Cruise control center. He said representatives from Cruise responded to firefighters and “immediately disabled the car remotely.”
More seriously, the statement cites four specific clauses that caused the permit to be suspended. The second one seems really interesting as it implies that there was a lie of omission where the others are more or less what I’d expect regarding safety risks:
> Today’s suspensions are based on the following:
> 13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
> 13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
> 13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
> 13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
This looks like Cruise's fault, dragging a pedestrian while pulling over for "safety"?
The thing about trains is the rails are only made for trains.
I don’t think it is a good idea to mix human and machine driven cars, especially in city roads where there are cyclists, pedestrians, trucks.
I would love to see AV only lanes restricted to geofenced parts of the cities.
We have been conditioned to think of autonomous driving as the answer to the traffic nightmares in every city, but tech will never ever be able to solve a social problem (just like how building a wider road will never ease congestion). Had even a fraction of the hundreds of billions of dollars that have been poured into self driving so far been spent towards, say, improving sidewalk quality, making traffic lights smarter, putting sensors along roads, coming up with a standard communication protocol for these sensors, building out public transit, improving urban planning etc. over the last 15 years, we would all be living better lives today. But a VC would rather set that money on fire to have a small chance at a 100x return.
So in that sense, driverless cars will actually help because of kind of nuanced reason. My guess is that there will only be a small number of driverless car companies (tesla, waymo, cruise, apple, etc) and it'll be much easier to add regulations that those companies need to send their destination data in realtime to the traffic light infra. I think that would basically be a non-starter for peoples personal cars.
Perhaps for the optimum solution, but you can get a _lot_ better (than things are at the moment) without that. There are plenty of cities that use sensors and cameras to adjust traffic light timing throughout the day, and have done for some time (In the 90s I studied the system used in Sydney, which had already been in existence for some time then). But these things cost money.
Also, you don't need driverless cars for destination data - anyone using Google/Apple Maps/Waze, or even any recent car's inbuilt nav system can provide this.
And I reckon that even with the optimal solution in this regard, you'll still end up with traffic jams, as individual cars take a lot of space. Start thinking in terms of mass transit, cycling, and city planning to allow walking to the shops.
Self driving car technology could be (at great expense) be used on all the cameras in a city to get a better picture of where people are really going and thus turn lights green to cause the least waiting.
Most cars sit at a light burning fuel and thus emitting CO2. Then they use a lot of fuel (read CO2) to get up to speed and at the next light that fuel is turned into heat by the brakes (hybrids can do some recharge, but they are not 100% efficient so there is still a lot of CO2 for no work)
Blaming VCs for public policy failures is fair when they lobby to create those public policy failures. In this case, it seems like their efforts (lighting money on fire) are likely to improve pedestrian safety, so it doesn't seem fair to blame them.
It’s not like Cruise / Waymo could change city planning against the cities will, even if they wanted.
Cities have very weak abilities to raise revenue, and are relentlessly politically attacked whenever they try to raise more revenue via higher property taxes and other things.
Our cities are actually starved of revenue and especially post pandemic at risk of bankruptcy without bailouts from higher governments.
I don't know where this mantra came from but it definitely falls in the risible category of leftist rhetoric. Technology has already solved multiple social problems, most obviously the advent of telecommunications itself solving the problem of coordinating actions at a distance. But more embarrassingly, the solutions you propose in the following paragraph are mostly technological.
People just need to drop this line.
This is an infrastructure issue. Culturally I sort of understand how the US ended up with bad walkability and bad public transport. But what I do not understand is how road infra is so horrible.
Where was this mile-long stretch? 3rd street?
Many (most?) intersections in the Netherlands have early detection systems that change the light _before_ the car reaches it.
But they didn't just do this for cars, they also do this for bikes. And public transit gets signal priority allowing buses and trams to almost always get green lights.
They've spent the money improving transportation of all forms, not just cars.
[1] https://youtu.be/knbVWXzL4-4
[2] https://bicycledutch.wordpress.com/2016/06/21/traffic-lights...
So, the entire amount of VC dollars that went into Waymo is ~$5.5 billion dollars.
If you look at the entire self-driving market, it's probably less than $12 billion in total.
San Francisco's yearly budget, is $15 billion dollars. The Federal Gov spends 50 billion dollars per year on maintaining federal highways.
So yeah, you didn't have an epiphany. You just couldn't be bothered to spend 20 seconds thinking about the math.
[1] https://www.bloomberg.com/news/articles/2023-02-23/in-nyc-su...
Context: When sitting in the backseat of a Waymo, there is a tablet-sized screen that shows you all the cars nearby. And humans! On one of my first trips (at night), it took me a while to figure out what the white circles were. And then I looked over and saw people in the shadows! It was honestly kinda spooky that the car was much more aware of nearby humans then I was...
Disclosure: Googler, and a big fan of Waymo
> The [human-driven] Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.
Are you saying that they should all be equally safe? If so that would require a level of oversight and control (plus knowledge share?) not typically seen in US government, no?
I agree in principle, I’m not defending them at all. However this is exactly what much of the US wants, given how much de regulation and reduced “red tape” is often argued for.
Until then, we’re going to get some companies that put everyone’s lives at risk. Hell Tesla alone has over promised to insane degrees (the Full in FSD). I don’t see companies having less power any time soon.
https://www.vice.com/en/article/4a3ba3/california-dmv-suspen...
One wonders WTF they were thinking. Were they so sloppy they just forgot to show the video? Were they hoping DMV would never see it? Either way it's a terrible stupid and short sighted mistake. A major problem for Cruise is convincing governments their technology is safe and responsible. Hiding evidence is not going to help that case.
Uh, 'Mistake' is a really generous word here.
Cruise isn't some amateur one-man startup. It's a multi-billion dollars company with tons of lawyers, advisors, resources, support from GM, etc. It was a deliberate act full-stop. The only 'mistake' they made was probably thinking they could get away with it.
Usually when private companies are developing products that could kill, drugs for example, it is on the company to exhaustively prove the safety and effectiveness before launching it to the public. It seems odd that self driving cars have been skipping this step.
It's just that, in practice, it's almost certainly impossible to reach safe self-driving cars without a period of 'public beta testing', you'll just never get enough data and experience.
Not entirely unlike how there's a period for new drivers where they really aren't safe, but there's no way for them to become safe without going out onto the regular roads and gaining more experience.
[Citation needed]
"It seems odd that self driving cars have been skipping this step." is an opinion referencing well-known current events. You can't expect everyone commenting on this article to re-reference the saga of Cruise in San Francisco; especially when it's frequently covered on the front page of Hacker News.
Furthermore, you should know that a Hacker News discussion is not Wikipedia. These discussions are not the kind of things where every off-the-cuff statement needs to be backed up with references.
Its also a USian obsession. Some other places have been able to actually invest in superior public infrastructure but the US is so focused on techno-solutionism it, as usual, misses the extant and superior solutions. Self driving does nothing to mitigate the sprawl and atomization that wreaks havok on our society, but trains and busses, and density that cannot be achieved with car-dependence does.
A part of the sales pitch for AVs is that autonomy acts as a safety layer.
As with any sales pitch, the interest of the manufacturer is often more in getting people to believe the pitch than in it being true, validating that it is true is a concern of outside parties. Insofar as AV adoption requires loosening existing rules to allow them, that includes regulatory authorities.
Here is Cruise initial NHTSA filing: https://static.nhtsa.gov/odi/inv/2023/INOT-PE23018-11585P2.p... They mention every detail from start to end but don't mention the fact that they dragged a person under the car.
Let me be very clear, if Cruise just collided with a person due to the mistake of another driver, that's cool. However, this is something way worse. Cruise car stopped, and then decided to drive 20 feet dragging the pedestrian under the car because someone on Cruise team designed a rule that says "if there is a collision, move away from the spot after stopping to avoid blocking traffic". No human driver will ever do that.
They are testing their garbage in dense urban areas. Too many times their vehicles just stop in the middle of the road.
If it rains, vehicle stops.
If the light is out, vehicle panics and stops.
Somebody puts a cone on the hood, vehicle panics.
If a single variable not accounted for in their algorithms shifts beyond expected range, it fucking panics.
So much money dumped/wasted into “autonomous personal vehicles”. How about we invest in making our cities more pedestrian, bike, and public transportation friendly? Let’s move people more efficiently rather than further digging ourselves into this hole.
I don't hate public transit or anything, I cycle and subway in NYC all the time, but the level of investment the US needs in public transit is an order of magnitude above the amount that investors have spent on autonomous driving.
[1] https://www.bloomberg.com/news/articles/2023-02-23/in-nyc-su...
There are tons of companies that would love to help (get paid to) build out public transit infrastructure in cities. You think any major city would have trouble attracting bids to build tram systems, subways, or even buy more busses?
Look at your local leaders, how many of them are pro-public transit? How many of them have a long history of blocking projects because they want to "preserve neighborhood character" or some other lame excuse?
A developed country is not a place where the poor have cars. It's where the rich use public transportation.
Alternatively if you reject the premise of those questions that this is a necessary phase to pass through to get to the other side, what do you think the impact on progress would be if no "real world" training opportunities were allowed for these companies? Do you think they'd be hampered and progress would take longer, and what would the impact of that be?
Or more succinctly: "You can't make an omelette without breaking some eggs. Sorry, eggs [regular people].
Ends justify the means thinking leads to some dark places (e.g. what kind of medical-technological progress would we be able to make if it was possible to do testing on humans like we do on animals?).
The problem in this case happens to be that the AV tried to pull over instead of stopping in the middle of the road, so not sure this is a great criticism to make.
Cars also don't only exist in dense cities. They're even used, among other things, to get people into and out of those cities
[0] https://twitter.com/Cruise/status/1716877217995894934
[1] https://www.sfchronicle.com/bayarea/article/woman-run-autono...
You could, however, argue that staying in place was the right thing to do; continuing to drive to get the woman out from under the car may have injured her further. A follow-up article[0] suggests just that, and emergency crews ended up lifting the car off of her instead of trying to tow it.
If this is indeed why their permit was suspended, it seems like a questionable reason. Some of the other dumb things Cruise cars have done seem much more obviously egregiously bad. But in this case the crash may have been unavoidable (by human or robot), and avoiding injuring her further by not unpinning her may have been the right move.
[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
I think the bigger issue is the decision to move the vehicle after a collision involving a person not in a car. I've not had the misfortune to be in such a collision, but in the collisions I've been in, I would get out, and inspect the situation before taking any further action. Moving out of traffic is a good thing to do, but especially if there's a person, it's important to consider the situation before further movement. Even if it were just a vehicle to vehicle collision, I'd want to check if someone needed urgent medical attention, and acknowledge the collision with the other driver, make a plan for where to stop to discuss, and check for and clear any obviously sharp debris that may pierce tires of my vehicle or others' vehicle. A lot of that is outside the capability of an automated vehicle... but maybe human review before pulling to a safer place is a good choice especially if a person outside of a car is involved in the collision.
It may not be the performance of the vehicle in the incident that led to the suspensions so much as Cruise's performance in the investigation of the incident (or information about Cruise's prior behavior on safety info that came to light through the investigation.)
So a pedestrian was crossing a crosswalk when they didn't have the green light. It doesn't really make sense why neither the human driver nor Cruise car slowed down in that case and kept driving normally. Both should have stopped despite having the green light.
It's highly situation specific, and I'd expect the emergency personal to make the judgement call on how to free the person (potentially by taking control of the vehicle, or other means).
They do! https://slate.com/news-and-politics/2015/09/why-drivers-in-c...
We should be suspending all bad driver’s licenses, AI and human alike. Today we give people passes for crashing while driving badly, and we should not.
This poor woman was involved in two accidents really, and it is fair to shut down Cruise until they address this failure scenario of driving on top of a victim in road.
The first picture in the SF Chronicle article is harrowing: https://s.hdnux.com/photos/01/34/54/62/24300697/3/1100x0.jpg
These autonomous cars have done some truly dumb things, but I don't think this incident is so obviously bad.
Meanwhile, the human driver who actually hit her first, and then fled the scene, still hasn't been found...
[0] https://www.sfchronicle.com/bayarea/article/woman-run-autono...
"When the AV tried to pull over, it continued before coming to a final stop, pulling the pedestrian forward," the company said.
While not impossible, it's hard to imagine a human-driver scenario that would look quite like that, regardless of how legal accountability might resolve. Would a human driver stop their vehicle on top of the person they just struck? Would they then ignore the person they'd trapped and wait for a bystander to intervene?
I'm not suggesting that this scenario necessarily warrants the suspension, or that most of the same problems might not occur in some human-driver scenario, but it points to some of the subtle differences of human vs automated drivers that come up in exceptional situations. There are a lot of exceptional situations when driving and it very much matters how automated processes navigate them.
"“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”"[0]
[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
It's not hard to imagine that it's simply unequipped to know something like that, and it does seem fair to require that it become so equipped.
Huge supporter of autonomous cars, still feel like this is completely reasonable to require a solution for.
Updating my comment to include this:
“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”
https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
Society is not a monolith. The people and companies choosing to invest into self-driving cars are not the ones who would be able to make communities more pedestrian- or transit- or bike-friendly. As much tech as there is in the bay area, Google is not a local duke that can unilaterally make streets less car-dominant.
The people who can choose to make communities more friendly to other modes of transportation are the citizens who vote for representatives and laws, the citizens who show up at local meetings, and very often they fight against safer streets, sadly enough. Democracy is better than non-democracy, but that doesn't mean voters always make smart decisions.
Personally, I would LOVE it if American streets looked more like Dutch streets, or even German streets. But within US politics, wanting safer streets like those is an extreme position. Most people grew up with car dominance, and they're uncomfortable with or skeptical of the idea of actual choice in transportation modes. There's an enormous amount of cultural momentum behind forcing everyone to drive.
Cars were here yesterday, people loved them for their freedom. Cars are here today, people love them and build communities around them. Cars will be here tomorrow, and people will still love them.
This is a political problem though, it's not something businesses can solve.
You can be taken exactly where you need to go in much more comfortable conditions with much less risk of running into anti-social behavior.
That's why Uber has a market even in cities like NYC or DC.
I’m an environmentalist and the solution is electric vehicles powered by renewable electricity. It’s not cooping people in tiny apartments and foisting dense living on people that don’t like it.
Bikes? If you mean e-bikes and motorcycles then maybe. Buses and trains? No it's not and it will never be. This is a position only a green extremist would take.
A fucking horse is better than any of those methods of transportation. There are two crucial elements a car and a horse have in common:
The first element: You own it (or rent it) and you tell it when and where to go. The when and where part is crucial.
The second element: It has the capability of carrying not only you, but also an another person and cargo.
The only transportation devices that can do both of these things are a) cars, b) horses, c) bikes (kind of... two people and cargo on a bike would be pretty tough to do, let alone with any level of comfort).
The fundamental, unfixable problem with trains and buses is that you don't tell them when and where.
The future is in EVs or global warming. Personally I've had it up to * here * with all the green hypocrites so I say bring on the warming, let's fuck this place up if you don't want electric cars.
That said, I don't have access to the data and I don't really know if Cruise has been acting inappropriately. I'm glad that Waymo seems to be generating far fewer negative press reports, which seems to be a good proxy for whether regulators are going to clamp down. So I hope Waymo continues to operate successfully (and expands their service to me in particular) and I hope that Cruise improves their system and gets back online.
Cruise was pushing too hard and fast for where their stack is at and they seem to have some severe safety related bugs.
Ex [1]: The car randomly starts swerving rapidly across traffic lanes towards an occupied sidewalk.
Ex [2] (@1:31): The car doesn't stop for someone already in the crosswalk and nearly hits them.
[1]: https://www.dailymail.co.uk/news/article-12640657/self-drivi...
[2]: https://abcnews.go.com/GMA/Living/video/driving-taxis-hittin...
That's a common response from people who would prefer NOT to know the facts.
https://getcruise.com/news/blog/2023/a-detailed-review-of-th...
"The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet."
First of all, I find it ironic that the human driver is still at large. I'm sure the Cruise car recorded the plate and they passed this to the police.
Second, the corporate speak on the Cruise release is very good. Notice how it says "pulling the individual forward", in other words, the car dragged the person while it attempted to get out of the way. I guess nobody thought about that edge case. And they released a completely irrelevant simulation showing that if the other car had been an AV this wouldn't have happened. Yet, the real issue with the Cruise AV is that after a collision it just blindly tried to pull over and it dragged the person. The same thing would have happened if the person had simply collided with the car on their own.
Jesus Christ! Good riddance, I'm glad their permits are revoked. Can you imagine witnessing this and watching an autonomous vehicle slowly crushing and dragging someone underneath? That 20 ft must have felt like forever to witness. Let alone experience.
I welcome our robot overlords.
I'd say the initial hit probably did a chunk of damage to the pedestrian, but quite likely not fatal unless their head hit the ground. But then being driven over and dragged 20 feet is going to be pretty scary and in all likelihood cause more serious injuries than the initial impact.
“The Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.”
So there was this woman crossing a crosswalk in front of these two cars when she shouldn't have. I guess I don't understand why neither car stopped in front of the crosswalk when they saw this woman in the middle of the crosswalk (illegally).
IMO it was a bit aggressive for Cruise to have seen this woman jaywalking and still crossed the crosswalk after detecting that the woman had just passed the car but still in the middle of the crosswalk.
This sort of nonsense where you take a serious statement and tell me the cutesy "name" of the autonomous vehicle really detracts from the seriousness of the issue. Also, the last paragraph where they try to talk about how their vehicle wouldn't have made the same mistake as the human driver, glosses over the actual issues the cruise car had which a human driver would have known not to do.
so pulling over after a collision, which resulted in dragging the human 20 feet is an interpretation of this regulation.
sounds like California and Federal AVs regulations are equally not safe for the public's operations
which is a good enough reason to revoke permits too, until they fix this together with the private industry
That’s a pretty enormous screw up and not one I’d heard about when this was initially reported. Interesting the statement from the DMV implies they were lying…
Having ridden in Cruise cars a couple of times, I am glad I was able to ride one while I had the chance. The novelty factor was very high, and while being hard to hail + very slow, it was a great thing to show to visitors to the city. Hopefully these safety issues can be resolved. As a biker, I feel much safer around a cruise than your average prius.
At least the autonomous vehicles actually follow the rules: they are patient and always watching the bike lane and crosswalks. I've never had one try to cut me off, make an illegal turn, or aggressively turn into me while in the crosswalk. These things are daily occurrences for me with human drivers due to the sheer lack of traffic enforcement.
Waymo has been very cautious. This turned out to be much harder than expected. Waymo's stats get better each year, and it looks like this is going to work.
Later, the car in front of me was going down the hill and my cruise car kept accelerating and then braked very hard once its sensors were able to see that something was in front of it.
It was an okay experience, but I’ll be honest, I don’t think I would be willing to take myself in it again or my family anytime soon. And honestly, I can’t believe that moped driver didn’t get hit/I didn’t get whiplash.
Before this, I was really impressed by Cruise and actually bought shares in General Motors because I was so blown away. But yeah those cars aren't perfect. However, maybe a human was controlling the car when it blew threw that cross walk? I would not be surprised if a person were to do something like that because people drive horrible all the time, but I would expect better from a computer.
Maybe that's part of the problem? We have bad drivers who are ultimately writing the software that controls the cars.
Anyway, those are my thoughts. If someone from Cruise happens to read this, try and add some code to prevent these cars from blowing through cross walks with people in them. (I'm not sure what the law says about this, but my interpretation is that if the cross walk sign is lit up, and there's a pedestrian in the cross walk, you should stop. Maybe it's okay to go if they haven't made it to your side of the street yet, but you should never get so close to a pedestrian that they can strike the vehicle out of anger with an object.)
At all.
They are classification systems. They have no understanding whatsoever of anything around them. Which means it is likely that the AV which is the subject of this accident didn't understand that it hit a person, what a person is, how a person could be stuck under the car, how a person could be dragged by a car, how a person could be hurt by a car, how a person is more fragile than a car, why it might be better to not move in any of the above contexts, how a human driver would more likely get out of the car and make sure it was safe to move (and render first aid), etc.
We are calling this crap "Artificial Intelligence". It isn't. It is not at all. It's a parlor trick devoid of anything that resembles understanding. A dog understands the world far better than the most advanced self-driving-car.
This lack of understanding issue carries with it layers of significance. For example, a human driver understands consequences to his/her actions. Yes, of course, there are a-holes in every crown...yet, the vast majority of people are good. If that were not the case society would not function as it does. Lacking the concept and understanding of consequences, just assigning scores to things and path planning accordingly, removes important decision-making layers and drives suboptimal AV decisions.
You can add responsibility, fear, financial consequences, emotion, care, consideration and a whole host of human condition realities to the list of important factors completely lacking from AV's today.
Driving is fine when reality is reduced to easy-to-classify monotonically varying experiences. Not so easy when reality becomes, well, real.
I don't understand why authorities allow self-driving cars to beta test out in public. I thought Waymo was the best of the lot with something like millions of hours on the road and no incidents, but when they started accepting passengers, it failed to proceed past a traffic cone on a sunny day:
https://youtu.be/zdKCQKBvH-A?t=742
The government should initiate a comprehensive test on a self-driving car's capabilities. In their current form, I wouldn't trust any of the others like Tesla and Uber that think self-driving is as easy putting it out on the road. And that too with Tesla and their cost-saving no-LIDAR nonsense.
These companies should be fined hefty amounts and barred from beta testing out in public.
It does feel like a lot of these weird/longtail situations should be more quickly resolved with some better combination of letting a human override in person (like when a AV car blocks a firetruck) and/or quickly getting a human to remotely take over, but yet we keep seeing these reports of where the car does something dumb for a long period of time. Is it as simple as they don't have enough employees on-hand to intervene?
I've been writing my distaste for Cruise for the past few month or so here, because of this. Of course getting downvoted by those with skin in the game but c'mon you can't act like how Travis Kalanick did...you are pushing technology onto the streets that can kill people! As well Waymo (Google who im no fan of) has been working on their robot car tech since 2007 ... Cruise no way for that long.
There were reports they are pushing their tech onto many city streets. Sorry to say and hopefully other cities/states follow CA's lead!
I realize assuming this is the California DMV's line of reasoning is speculation, but this sentence from Cruise's own public statement on the event really does it for me (emphasis mine):
> The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet.
The tricky thing about these vehicles' control software is that it has absolutely no judgment or common sense. That's really on display here, where the vehicle immediately goes into a canned program instead of following a course of action that takes into account all the details of the situation in which it found itself, not just the ones that its creators had anticipated.
That doesn't necessarily mean that testing self-driving cars on the roads is inherently irresponsible, but it does mean that the ground rules for how we manage such programs need to be tailored to the situation. What we have here is a situation where a rather nasty software defect has been discovered. We can assume that this defect is present in all of the vehicles Cruise is operating on public roadways. It is therefore reasonable and prudent to temporarily suspend autonomous testing operations on public roadways pending analysis and remediation.
Because, let me emphasize again, TESTING. Pausing to analyze and iterate when you find problems is an essential part of testing. It's arguably the main thing that distinguishes testing from just YOLOing shit into production.
Fixing all Cruise vehicles so that this doesn't happen might be hard, but it isn't impossible.
Accidents...manageable.
Hitting a pedestrian...still manageable.
Deliberately withholding evidence....burn them at the stake and regulate them to an inch of their corporate lives.
Tesla is going to smoke these companies. Theirs is the only software trained on high quality actual driver behavior (which they can pull from their fleet because they have telemetry in the form of a real-time safety score for every driver).
It's like every other company is trying to write an algorithm to construct strings, and Tesla has GPT-4.
They don't have the hardware to compete with Waymo and Cruise.
But sure, it is a good narrative for the stock to go to the moon.
We, I mean humanity, need an example that L4 self driving is possible.
When SpaceX showed that private company can access space and build a product in the area world changed.
Startup founders, investors, government changed their opinion and many space companies were created and trying to do things that they would not even consider before SpaceX: Astra, Rocket Labs, Planet Labs, Firefly, etc
(Great book on the topic "When Heavens went on sale": https://amzn.to/46NRx9l)
If we get one example of self driving as a product on the roads, it may have the same effect. We are going toward self driving winter and unless Cruise, Waymo or similar companies will do the impossible new iteration will happen in 10+ years.
It is mentioned in the text that in Q3 GM spent 700M on Cruise.
Not that many companies in the world: 1. have similar level of talent as engineers at Cruise 2. can spent similar amount of money 3. are willing to spend them
And we need all three.
There are so many corner cases that without large scale as close to the real life deployment as possible...
I feel very sad.
P.S. Worked for 3 years in a self driving company, till our division was sold :(
This was at night with zero traffic. I have no idea why it took so long.
When the CPUC was set to vote on expanding Cruise’s operations several months ago, SF first responders and other residents spoke out against expanding operations. CPUC ignored all the feedback and concerns and voted to expand. I believe the press pointed out the CPUC people that voted to expand Cruise operations would benefit off the expansion (can’t recall if they were former employees or tied to the driverless industry). But despite emergency responders giving numerous examples of how dangerous the cars were in emergency situations, they were ignored.
As far as I can tell, they were open in 4 metro areas (SF, Austin, Houston, Phoenix). So this is their original testing zone, and 1 out of 4 of contemporary zones down.
Will this impede their ability to retain and expand existing zones?
What I found so far is this:
- Waymo and Cruise have driven about 8 million miles driverless (mostly in San Francisco) and reported about 100 crashes (mostly minor like scraping shopping cart, etc) and many that were fault of another car. (as of September 2023)
- Apparently a human drives am average 100 million miles between fatal accidents. In that case we do not have enough data, yet.
I'm not sure what to think. Is the rate of accidents a cause of concern? Or is this blown out of proportion and driverless cars are actually safer? Or (most likely) we simply do not know, yet?
My drive to work is 20 miles and a really good stretch of that is just 1 lane where I’m stuck behind a car or the road is empty. In those cases I have no problem letting FSD handle everything, the road is extremely hilly, bendy, lane lines vanish every once in a while and it’s fine so I do find it useful at times, though not worth 200$ a month by a long stretch. I’ll probably cancel the subscription soon.
Its problems appear very clearly whenever it needs to accurately predict what other cars on the road will do to make its decision. It is honestly horrible at that and while a few times I tried to not intervene, I realized I was just creating a nuisance and now stopped allowing FSD to deal with those situations. It is also bad if there are wrong lane lines (happens at one final stretch of my daily drive)
Finally, as someone working in AI, I’m actually very optimistic about their next approach of just learning an end to end model to drive the car. I think this is the only scaleable way to reach human level driving. Any set of rules based + optimization based planner will have those edge cases that ruin the cars capability, in fact it turns out the long tail is really long. When a human is not drunk and focused he is a remarkably good driver. On the other hand, if you just need to capture the edge cases data and add it to your training set to solve it, you can capture all the possible driving conditions that come up on road. This reminds of how there were hundreds of failure modes and jailbreaks for ChatGPT early on while now there are close to None, presumably because they just added them to the training set. If a good human level self driving car appears, I’m convinced it will be an end to end approach, whether the Tesla team are capable of this, who knows.
Also for those who think, OpenAI’s scale based approach only works for language, I have access to ChatGPT Vision and its capabilities astound me. Scale for sure works
Tesla doesn’t have an end-to-end model. They call it “end-to-end AI”, which just means they’re applying AI to all subsystems, exactly like everyone else. All the big players already use heavy ML-based planning systems that are not “rules or optimization based” and which are way more sophisticated than anything Tesla has put out or is planning to.
Why aren't they investing in core engineering, instead of moonshots?
> In a statement, Cruise spokesperson Hannah Lindow disputed that Cruise failed to provide the full video during the first meeting with the DMV. “I can confirm that Cruise showed the full video to the DMV on October 3rd, and played it multiple times,” Lindow told Motherboard in a statement.
Filling out forms with made-up bullshit just to get an approval works great when you are putting together a SaaS sales presentation, not so much when it's the state Department of Transport on the other end and you are working on matters of public safety.
Is there something specific that caused this halt on Cruise's testing permits?
If that car gets found doing something obviously illegal (speeding, running a red light, some kind of "reckless driving" camera review, etc) the registered owner is automatically assigned blame. They can potentially file paperwork to the court to prove someone else was driving (a lease arrangement, some valet ticket along with some proof from the restaurant or whatever at the time, a sworn statement from another individual confessing to it, etc). But ultimately by default the owner of the car is responsible for the safe handling of the vehicle. If you're worried someone is going to break the law with your car, don't give them your keys. If you don't like this, find another form of transportation.
Tag enforcement needs to be stronger. If an automated system sees a tag that doesn't make sense, it should be immediately flagged and enforcement dispatched. Lots of cops already have ALPRs, those should immediately trigger. I see so many cars out there with sketchy paper tags, this whole thing needs to be shut down.
Driving a car on public streets shouldn't be nearly as easy as it is in the US. Far too many people don't treat driving with the respect it needs.
10000% we need safer designs for streets and roads, better education, automated enforcement of speed limits, dis-incentives for larger/heavier/more dangerous vehicles, ...
All of these things need to be done, and can be done with existing technology today. We don't need self-driving cars to address our problems; they're essentially a distraction and an excuse to postpone improvements.
Why would you let, by definition, more dangerous cars on the road in order to perfect their software, when perfectly good solutions exist?
Perhaps in the end Waymo will be forced to license the base technology somehow.
Understandably it is expensive. But the profits will similarly be great.
They ought to build their own mini cities to test on as part of their own Manhattan Project.
First, the California DMV, with result seen here.
The incident that triggered this event happened when a human driver, who is still at large, hit a pedestrian who flew into the path of the Cruise vehicle.
The car executed a maneuver to pull over for a safety stop, but failed to recognize the woman who was dragged some distance.
Second, they reached to the Federal agencies via Nancy Pelosi’s office, with primary focus being the NHTSA, the National Highway Traffic Safety Administration.
The request was to gather data, which given the friendly posture of the NHTSA at this time will likely result in the opening of an investigation.
Both of these moves have some bite. The California DMV can suspend licensure as happened here.
But they’re fairly straight-shooter civil service types in the end, and they’ll eventually clear the vehicles for use. You can see this in part because of the action against Cruise only, rather than Waymo.
Some may attribute this to Waymo’s greater political sway in California. But my experience of the products themselves is that Waymo has a higher quality self-driving system.
The NHTSA is a bit trickier to predict.
The NHTSA’s powers are broad in theory and as granted by statute. They could theoretically issue an order to remove a type of vehicle from the road for imminent threat to public safety.
However, their typical modus operandi is to issue recalls and work with the auto companies. They won’t want to test these powers under the current court, so my opinion is that they won’t take much action.
After leaving the state, I'm glad to not have to drive on the same road as these.
Yet, trying to create near-peer artificial human driving intelligence—just so cities can continue to be designed around horseless carriages, minivans, and truck nuts—well, that may not have been the best solution to the problem.
Ideally, autonomous driving would be for “the edge” of a transportation network. Like a house on a dirt road, or a camping site, or the home of a misguided friend who moved to south Gilbert, AZ ;)
You shouldn’t need to be watching for basketballs and red lights on interstate highways. Everyone is going the same direction for long distances between transfers. Cars are often individualism, which I’m certainly not against.
The arrogance of “driving driverless should be easy” probably comes from a lot of people seeing driving as a part of their basic humanity. The car is nearly a cyborg extension of their mobility and physical power.
Me? I just wanna get to the concert. Take the humans out of the equation where it makes sense. I’d rather go with light rail; not my friend who offered to drive, then road rages over a merge, then follows some minivan psychopathically for a few minutes (true stories).
We shot ourselves in the foot because mass transportation in America has failed to impress us as much as a Jaguar XK or Jeep KJ, or whatever your Soul Car is.
As much as I'd like to agree with this, the fundamental problem is that most people in the US do not work at a single point in the city anymore.
That's the real problem. You don't get to connect existing enclaves. You have to build your mass transit, fund it at a loss for decades, make sure it doesn't get overrun with crime, and maybe after 10-20 years there will be enough housing and office space along the line in order to make it viable. Maybe.
Or you can throw down a couple more roads and let people figure out where to live, themselves.
[0]I can see myself not agreeing with the conclusions, but to not even see the question broached in these terms is maddening.
No advanced robotics necessary.
Note for example today's suit against Meta for alleged harms against children. No such suit against TV aimed at children or e.g. fashion magazines that are well-known to have deleterious effects on children. Autonomous vehicles are new tech, so the bar will be higher.
I think AVs need to be better than p95 or so to be socially accepted.
is it accepted as a standard by any professionals? Why or why not?
But every new model or software update will invalidate what we've observed and know about automated driving.
We're in an era of extreme obfuscation and enshittification by tech companies, even where safety is involved. I'd be more likely to trust self driving tech if I trusted the current form of the tech industry, and trusted that regulatory bodies had the power and ability to oversee them as effectively as when they were first regulating cars.
Of course even that wasn't perfect, but my trust that we can even do that well is nonexistent.
“A highly-skilled and attentive human driver” is a reasonable standard. So-called autonomous vehicles are nowhere near that.
Those that do are making a costly mistake. Any robot-assisted car still requires a human driver to be paying attention to the road, but by definition, because the robot does most of the driving, the human failsafe is going lulled into a false sense of security, or bored by effectively driving without actually driving. They will slack, and when the critical moment of awareness from the human is needed, he will be caught off guard, as opposed to the actual human driver who was aware the entire time, because he always had to be.
More to the point, even if I had a 100% guarantee of safety, I would never want to have the joys of driving taken away from me in the first place! Driving is liberating. It's power. It's the freedom to go anywhere and do anything. It's, dare I say, cathartic.
And for those that are unable to drive, Uber & Lyft exist to fill in the gap. They are themselves a relatively new technology, just under a decade old, and you can always find a quick, affordable ride on demand to just about anywhere.
What gap in the market would a self-driving car even fulfill?
This is yet another example of the arrogance of big tech, pushing out unsafe and socially deleterious products in the hopes of a get rich quick scheme. It's no different to what social media did a decade ago or what "generative a.i." is doing now.