How is an autonomous car going to prioritize lives when someone is sure to die in a situation?
Tom Allen's answer is an excellent response to the literal question. But the question itself is flawed, despite the fact that I probably come across ten articles a month posing this dilemma.The dilemma is not real.The vast majority of today's car collisions:A) Do not happen "suddenly" enough to warrant such a dilemmaB) Are not the kinds of incidents autonomous cars will createI challenge anyone to find, in any legitimate study on traffic and collisions, a single scenario which registers as "sudden" on the scale of an AV's computational speed. Short of that, the notion of powering artificial intelligence with human decision-making processes in order to solve for faulty human behavior is stupendously short-sighted. AI doesn't think like us.A. Collisions are not suddenRecall a time in your car when you had to think and react extremely quickly. Nevermind whether you made the right decision or not, how much time do you think went by between the moment the scenario occurred and the moment you applied useful force on the pedals or steering wheel? Chances are it was well over 1 full second, even if you were fully attentive and had above-average Driver Reaction Time. In most cases, it takes longer than that... and remember, here we are only talking about scenarios where we assume that there was no prior information available in the environment to plan for or mitigate likelihood of the collision, which I'm still waiting on someone to present examples of.There are always environmental clues, and even if there weren't, there is precedent (which is why Google's cars wait a second after the light turns green to proceed into an intersection. Bad things happen during the transition from red to green). The frequency, volume, and speed of processing power these AVs have already -- let alone will have -- obliterates any notion of a "sudden" event. The upshot is that the OP's dilemma will not exist, because the vehicles will not place themselves in those situations. Show me a collision situation that appears instantly and requires no greater than a few milliseconds' reaction, and I'll show you a problem that is too fast for AI to process -- which is Tom Allen's point. None of the ethics we might impart on AI could be recalled and applied in time... and it doesn't matter anyway, because to humans, the event would be instantaneous.B. AVs will not get into these kinds of incidentsDebating what self-driving cars will do during impending collisions assumes collisions are indeed unavoidable accidents. But, as I've laid out above, what we call accidents are actually just mistakes, and usually a series of them over a very manageable timeframe. The incidents we know today as collisions will not happen (at least, they won't be caused by AVs... I can't speak for the dozen or so humans who have hit Google's cars http://www.google.com/selfdrivin...)Autonomous vehicles will get into two kinds of incidents: one of them is a collision that is unrelated to the OP's dilemma, and the other is a non-collision that is related to the dilemma.1. Overfitting/malfunctions/etcThese are the ever-decreasing scenarios wherein an AV will fail to acknowledge that an incident is materializing. If you look at the vehicles from DARPA challenges of years past, you see examples of this. It's not that a car is morally unsure of whether or not to hit a mailbox, it's that the car doesn't see the mailbox, or sees it and believes it to be open roadway. Only when it senses the actual impact will it realize an anomaly has occurred. These issues are almost entirely technical, and the only moral dilemma is whether or not we get up in arms as a society about a technology that simply ran over a kid who was wearing all black at night and lying down in the middle of the road on the downslope of a crest. But even here, the logic from Tom Allen's answer applies: 85 people die every day in car accidents. The more time society wants to spend trying to analyze and solve for the 1 kid who might get killed by an AV (in a situation it learned from and taught to the rest of the vehicle fleet, no less), the more people die in the meantime. So, if the debate folks want to have is what to do when an AV malfunctions, that's fine... but just know there is probably more social capital being lost in having the discussion than by just letting it go.2. Currency-based decisionsA Self-Driving Car Might Decide You Should Die — BackchannelThe biggest ethical dilemma in autonomous cars gets zero coverage in media or academia (the above is my own article, apologies for the shameless plug.) It could involve collisions, but most of the time, it will just involve the everyday interactions you take for granted.People claim safety is priority #1 for autonomous cars... but it can't be. The AI we see being used today gets by on such a priority (i.e. when in doubt, stop the car) only because the rest of the people on the road are at least mildly interested in following the established rules.Now, place an entire fleet of autonomous vehicles on the road. All of them are saying, "when in doubt, stop the car." If you're a pedestrian waiting to cross an intersection, and you know the entirety of traffic will stop if you step in front of it, wouldn't you do it? At least some of us, some of the time?There are many scenarios like this that don't currently exist, but if you think of anything you don't do in traffic today, it's likely you could do it tomorrow. And that's why safety can't be P1... anything could bring the entire traffic grid to a halt by refusing to yield right-of-way. A kid on a skateboard, a gal running to the meeting she's late for, a dog who got loose, a baby stroller that rolls into the street, etc.Why doesn't this happen today? Because we have a mental model of currency. I don't run out in front of a busy intersection because the reward doesn't outweigh the risk. If I felt otherwise, there's a good chance I'd get hit due to a driver's currency: that being able to semi-attentively drive 45mph in a busy urban area for his entire life outweighs the risk of hitting some idiot who runs into the street.AI needs to have currency. That's the dilemma. How can humans "inject" moral currency when a) we couldn't document it for ourselves if we tried, b) we all operate on different currency models, and c) our world view is so small that even if we could document it and did all agree on a model, the collective intelligence of AI would have a higher level of awareness than the human collective, and would therefore have moral problems to solve that we never imagined, nor understand?