Wow. The other driverless car players should be all over this lobbying to shut Uber down. If Uber massively deploys a commercial service with subpar quality in order to "win", and then those cars start getting into accidents, the entire field is going to be delayed by 10 years. The general public is not going to just think "Uber is bad", they are going to think "self-driving cars are bad". Politicians will jump all over it and we'll see very restrictive laws that no one will have the guts to replace for a long time.
And honestly if that happens, that's probably what we would need anyway. If the industry doesn't want to be handcuffed they need to figure out some really good standardized regulations on sharing data with law enforcement, how to determine fault for self-driving vehicles, and what penalties there should be. That are fair and strict.
The alternative, they are not first to market, someone else is and immediately replaces Uber with a network of cheaper self-driving cars: a) Uber goes out of business b) They can somehow convince someone to license the tech or sell them the cars at a reasonable price, making them vulnerable and less profitable, with no market advantage.
Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1].
> To get to that better software faster we should deploy the first 1000 cars asap.
Is that admitting that they do not have the "better software" and intend to deploy 1000 cars using "lesser software"? That's treading dangerously close to potential manslaughter charges if prove this willful contempt for safety to a court.
To play Kalanick's adversary, he might be arguing for more real-world data collection. Tesla famously equipped most of their cars with more sensors than were required at the time of delivery, using the data to drive development of the Autopilot function that was later added to the cars.
Btw. all this of collecting sensor data and improving on this, is an original idea of MobilEye that they presented at conferences before the so-called Autopilot was available from Tesla.
He is clearly right about that. Human-driven cars are safety critical and already do fine without redundant brakes and steering. How many crashes are due to brake or steering failure? I'm guessing it's well under 10%.
Most human crashes are due to bad driving, and for computers it will be the same. I mean, even this fatal crash probably could have been prevented with better software. It's not like the brakes failed. They just weren't applied.
> To get to that better software faster we should deploy the first 1000 cars asap.
This is where he is totally mad.
• Self-driving car kills child
• Hackers send self-driving car on wild ride
• Empty self-driving semi found in warehouse lot, GPS pirates make off with entire cargo
Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records.
But it will also drive up the cost.
And put it out of reach for a lot of people.
If the goal is to save lives, the bar self-driving cars should be held to is what humans do driving today, not perfection.
> But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States.
I don't think there's enough evidence to say that self-driving cars are as safe as humans.
Wow there goes that "safer than human drivers" argument.
Consider if there was a new lottery and you weren't sure what the odds of winning were. You play it three weeks in a row and the third time you win a million dollars. Conveniently, no one else tries the new lottery yet.
Does it follow then that the odds of winning a million dollars are 1 in 3? Or should you play it a few more times before you declare to all that one in three plays will make one a millionaire?
Assuming that accidents are independent, we can model this as a Poisson point process. If the accident rate is 1 per 100M miles and Uber has driven 3M miles, the probability of there being zero accidents in that time is P{n=0} = ((λt)^n / n!) * e^(-λt), where λ=1/100M and t=3M. Doing the math, it seems that's 97.04%.
So, yes. It is possible that Uber's accident rate is 1 in 100 million. If so, this incident would fall in that remaining 3%. It's unlikely, but possible.
Bayesian approach: To make the math really simple, let's assume a discrete prior on Uber's death rate. Say 33% that Uber's cars are much safer than humans (0.1 deaths per 100M miles), 33% that they are equally safe (1 death per 100M miles), and 33% that they are much more dangerous (10 deaths per 100M miles). After observing one death at 3 million miles, your posterior is should update to {safer: 1%, equal: 11%, more dangerous: 88%). This is a substantial shift in confidence.
Math: http://www.wolframalpha.com/input/?i=(1-10%2F100)%5E2*(10%2F...)
Frequentist approach: Let the null hypothesis be that Uber's self-driving cars have the same death rate as humans - 1 death per 100 million miles. The odds of Uber killing someone within 3 million miles is about 3%. Therefore, we can reject the null hypothesis with a p value of 0.03. One positive "data point" is statistically significant.
Statistically, one death after 3 million miles is not proof that Uber's death rate is higher than 1 in 100 million miles. But it is statistically significant, in both a frequentist and Bayesian framework. You have to get really, really unlucky to have a death at 3 million miles if your death rate is 1 per 100 million miles.
Bottom line: This collision isn't proof, but it's strong evidence. (To go along with all the evidence from crash rates, disengagement rates, engineers working at these companies, and the video of the crash itself.)
(I'm guestimating on some of those numbers, but the should be somewhere in the ballpark.)
They let a car on the road that couldn’t even stop at a red light ffs
https://www.theverge.com/2017/2/25/14737374/uber-self-drivin...
The fact that they were allowed to deploy in Arizona after this is really regrettable
And it’s totally unsurprising that Uber “got the first kill”
It only took two incidents to shut down the Concorde program.
I would add that people represent (as a whole) various kind of drivers, including the non-expert/fresh licensed, the elderly, the too sure of themselves, those speeding or not respecting road signals (or driving in connection with a crime), the emotionally fragile, those under the effect of alcohol or drugs, those tired by not having had enough sleep, etc., etc.
The "model" for an autonomous vehicle should be instead a particular "perfect" subset, ideally it should replicate the behaviour of an hypothetical extremely prudent, healthy 30-something with some experience with safe driving, very familiar with the way his/her car handles, having had a nice night sleep, no use of alcohol or drugs, without any external pressure (to arrive in time, to do other chores apart driving) without a cellphone or tablet distracting him/her.
If we could find this latter kind of drivers (should they actually exist) and isolate from the statistics the amount of incidents they caused, that would be the reference benchmark.
Taking it a step farther, I’d expect that road conditions in the Uber tests so far have been more benign than average city driving. Less bad weather, straighter roads, etc.
A proper comparison in this case is comparing passenger car death rates.
And then you need to factor in other conditions, such as the fact that weather was clean, and that you should be looking to compare pedestrian/bicyclist deaths, and you see that this incident already throws out wack the death rate for autonomous vehicles.
> I believe with perfect faith in the coming of the Messiah; and even though he may tarry, nonetheless, I wait every day for his coming.
It's a pretty unfair comparison, with 1 death on one side and over 30k on the other...
This is because stopped vehicles often have people come out of them to fix the vehicle, and too many cops, tow operators, etc, were getting killed by careless motorists.
Guess this is news to you (and uber), update your driving style please for the sake of first responders!
For example, unnecessarily stopping in a middle of a highway is extremely dangerous, especially if visibility is limited or roads are slippery.
Looks like Uber has attracted Levandowski due to his cultural fit.
"To get to that better software faster we should deploy the first 1000 cars asap. I don't understand why we are not doing that. Part of our team seems to be afraid to ship."
And from another email:
"the team is not moving fast enough due to a combination of risk aversion and lack of urgency"
And then, if their code results in a death, they are liable and can have their license completely revoked, and they would be unable to work on self driving cars again.
- If bad code makes it into production, that is a systemic failure not an individual one (Why didn't the bug get caught in code review, QA, etc.)
- No one is going to want to work on a project where a single failure can taint their career.
- What if I use a 3rd party lib and that is where the bug is. Who is at fault then? What if the code isn't buggy, but I'm using it in an unexpected way because of a miscommunication? If I am only allowed use code that I (or someone certified has written) development is going to move at a snails pace.
- What if I consult with an engineer who doesn't have a certification on a design decision and the failure is there, who is at fault?
- What if the best engineer on the project makes a mistake and ends up banned? Does he/she leave the project and take all their tribal knowledge with them, or are they still allowed to consult? If they can consult, what stops them from developing by proxy by telling other engineers what to write?
Not to be a dick, but this is an awful idea that would basically kill the self driving car.
In safety-critical fields, setting a much higher quality bar than the regular 'it seems to works, the tests pass' seems perfectly rational to me. We can now write provably-correct C compilers (CompCert) and OS kernels (SeL4). There's no excuse for not putting similar levels of effort[0] into something as safety-critical as self-driving cars.
[0]: Note that I'm not advocating for "provably-correct self-driving car software" (that may not be the right approach, as a formal spec is likely unrealisable), but find the argument that "it's ok to write buggy spreadsheets, so it's ok to write buggy self-driving cars" to be morally unacceptable.
Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply).
But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.
That is the reason, why certifications are for. The same rules apply for medical and other areas.
"The engineers employed by Jack D. Gillum and Associates who had "approved" the final drawings were found culpable of gross negligence, misconduct and unprofessional conduct in the practice of engineering by the Missouri Board of Architects, Professional Engineers, and Land Surveyors. Even though they were acquitted of all crimes that they were initially charged with, they all lost their respective engineering licenses in the states of Missouri, Kansas and Texas and their membership with ASCE.[22] Although the company of Jack D. Gillum and Associates was discharged of criminal negligence, it lost its license to be an engineering firm." - https://en.m.wikipedia.org/wiki/Hyatt_Regency_walkway_collap...
From what we seen so far the Uber car failed to detect an obstacle, we also had Tesla crash where the car did not seen a truck so it is obvious that there are some major issues that are not tested for. They need to have better tests and maybe get better security drivers inside the cars so they don't text on the phone on the job.
https://insight.ieeeusa.org/articles/professional-licensure-...
If you want error-free software you need a blameless culture based around process, not individual ownership of code. It should not even be possible for an error to be one individual's mistake, because by the time it hits the road it should have gone through endless code review and testing cycles.
Just like it relocated its testing to get a more "business friendly regulatory environment".
For a mostly topical example of such rules: https://www.dmv.ca.gov/portal/dmv/detail/vr/checklists/outof...
That'd be extremely foolish. And regardless of the dumb things the previous Uber CEO has done in the past and the big deal people are making over a $150 license, they have still hired some of the best engineers in the world.
You basically have to find the brightest-of-the-brightest to build AI... and Uber pays very well and puts plenty of effort into recruiting that talent.
Not to mention the massive PR and monetary risks that are inherent in killing people with your products. That would make any company highly risk-adverse.
https://jalopnik.com/safety-third-is-the-running-joke-at-ube...
1) They make it appear that Uber is a car manufacturer.
2) Even though Uber has not been determined to be at fault, the author seems to want to make it that way anyway.
I'd be fairly surprised if there's any real appetite at Uber to continue with this now. It was never anywhere near their core competency.
3 years ago Uber hired ~50 specialists from CMU to work on autonomous vehicles. I'd call that a core competency.
https://www.theverge.com/transportation/2015/5/19/8622831/ub...
Maybe not directly, but is Uber's current business model sustainable without some form of self-driving technology replacing their human drivers?
"Indeed, it's entirely possible to imagine a self-driving car system that always follows the letter of the law—and hence never does anything that would lead to legal finding of fault—but is nevertheless way more dangerous than the average human driver. Indeed, such a system might behave a lot like Uber's cars do today."
It doesn't matter if Uber makes cars that are technically not at fault, if they're mowing over pedestrians at a rate significantly higher than human drivers then they should never be allowed on public roads. People mess up occasionally. The solution is not an instant death sentence administered by algorithm.
1) The physics of driving isn't random, so it could be said that there are no accidents in autonomous driving, only oversights.
2) It would set a minimum performance level by making it prohibitively expensive to have a dangerous car. Those who test responsibly would have a low enough injury rate that they could deal with the risk by taking out suitable insurance.
3) It would provide a strong incentive to make the best car possible and not to take expedient shortcuts.
4) Over time automatically liability would become irrelevant if it asymptotically forces the injury rate to zero.
5) We have an historic opportunity to create a culture that will eliminate the danger of cars. It might have an increased short-term once off cost, but a huge long term payoff in the reduction of health costs and human misery. If we miss this opportunity we will be stuck with the long term cost of an industry that will be competitively driven towards poorer performance, potentially against the will of the majority of players, by the actions of a few.
If you enforce your right of way and kill somebody, that's manslaughter in my book.
They are on the road with conditions because what they are doing is somewhat experimental still. There is a safety driver for a reason that did not respond. A human driver may have collided but would have responded and potentially avoided a fatality (if not a collision). The benefits of autonomous driving completely failed on all counts in this case, which imply that being on a public road is far to early for Uber - suggesting some fault to lie with Uber or the regulators.
It seems unlikely that the Police will find any fault because they probably don't want to have to file a criminal charge against the driver, but that is who it would go against if there was fault.
There was a case some years ago in Boston where a subway rear-ended another one because the operator was texting. The driver was fired and probably would have been prosecuted if there had been fatalities. Taking eyes off the road for this long seems insane to me.
If you really cared about safety, there are far more immediate and impactful solutions then spending billions on self-driving cars. If they came out and said that they were doing it to make money or make driving easier, it would have carried more weight. But you just can't trust a word this company says.
* Flouted Taxi regulations
* Living in legal gray zones in regards to contractors vs employees
* Designed a system to avoid law enforcement
* Performed shady tactics with its competitors
* Illegally obtained the private medical records of a rape victim
* Created a workplace where sexual harassment was routine
* Illegally tested self-driving cars on public roads in California without obtaining the required state licenses.
* Possibly stole a LIDAR design from a competitor
Now their vehicle killed a pedestrian in a situation that the self driving vehicles should be much better than humans at (LIDAR can see in the dark, and the reaction time of a computer is much better than humans.)
Uber has exhausted their "benefit of the doubt" reserve. Maybe, they need to be made an example of with massive losses to investors and venture capitalists as an object lesson that ethics really do matter, and that bad ethics will eventually hurt your bank account.
Self driving cars are currently in that state where they're always in accidents but never technically at fault. When individuals have this behavior patter their insurance company drops them because if they're so frequently present when shit hits the fan they're a time bomb from a risk perspective.
Edit: meant to reply to parent, oh well.
edit: wow this triggered some people. somehow 'if they are at fault they should be punished' got interpreted as 'they are not at fault and should not be punished'
> But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States.
We have a sample size of 1, granted, but it's not looking very good. At the very least they were expected not to be less safe than humans.
The point is that it is to early to test this cars on public roads. The fact that self driven cars will kill less is just a hope, we may never achieve that(there is no proof we can do it with current tech) or it will take more time, and we need some actual numbers that are not tampered with before letting this self driven cars on the road, the security drivers seem to not pay attention so this car testing are a risk.
There have been cars on the market for years now that can detect a dangerous situation ahead, even in the dark, and auto-brake. If they haven't caught up to several-years-ago's consumer model of car, then what on earth are they doing putting these things on public roads?
It will be interesting when Volvo pulls back from that relationship, because even the uptodate emergency brake assistent should have catched up. Except Uber as deactivated those in favor for their own technology (because it is superior).
EDIT: German Automotive Club did tested full sized cars in 2016, which means decaded old technology in IT. The results where that the Subaru Outback was the only one detecting persons in the dark. But was very poor with biclysts. https://www.adac.de/infotestrat/tests/assistenzsysteme/fussg...
Maybe testing of autonomous vehicles should be done off public roads (at least at this stage of development).
The way it would work would be: the human is driving and the software is, at the same time, watching the driver and figuring out an action to take. Every time the driver's and the software's behavior differ, is logged and analyzed to figure out why there was a difference and who guessed better.
But the way testing is currently going on, it seems millions of miles are wasted where nothing happens and nothing is learned.
Some argume that it is okay, because it will decrease risks in traffic in the long run. This is not a valid argument to allow on road bug-testing, as there is a lot of medical research that we as a society don't allow because of ethical concerns, even some research where the risk of death is essentially zero. Applying research ethics to the Uber situation, Ubers vehicles would under no circumstances be allowed on the road until it could be proven that they were at least as safe as all vehicles already operating on the road.
So while the technique suggested might not work well to solve the problem of safe autonomous cars, the more dangerous alternatives should absolutely not be allowed.
Yeah, that doesn't work though. Basically because you would need to have an excellent situation representation to really understand the drivers' reactions to outside events. But that does not exist.
Perception and situation representation are key to mastering the driving task, and they both differ greatly between humans and machines.
This is the #1 reason I'm sceptical about self-driving cars becoming ubiquitous any time soon. Clearly they have potential advantages over a human driver in terms of not being tired or distracted, better sensors and better reaction times, but their judgement in any given situation will always be a function of some predetermined inputs. It's a brute force approach.
Until a self-driving car can recognise a pub door opening around throwing-out time where a drunk patron is about to stumble out into the road from a hidden position, or that it's about to pass a park and a nearby school just finished for the day so kids will be kicking balls around and running across the road to join their friends, or that the recent weather conditions make black ice likely and the cyclist it's about to pass doesn't seem very steady, and take corresponding actions to reduce both the risk and the consequences of a collision, it's going to take a lot of brute force to outperform an experienced and reasonably careful human driver.
In short, reacting to an emergency 100ms faster than a human driver is good, but sufficient situational awareness and forward planning that you were never in the emergency situation in the first place is better.
"win-at-any-cost" and "second place is first looser" (sic) do not cohere with safety.
