Imagine you are driving along a street when two people – a child and an adult – step out into the road. Hitting one is unavoidable but which do you swerve to avoid?
Now imagine the car is a driverless model, what happens then? What moral calculation can the vehicle’s artificial ‘intelligence’ make?
This is the scenario of Moral Machine, an online “serious game” that members of the public from more than 100 countries can take part in at moralmachine.mit.edu, to help decide how self-driving cars should ‘react’ when faced with such moral dilemmas.
It forms part of the world’s biggest ever psychology study. Researchers have devised it to gauge public opinion on the subject before fully driverless cars are on the market.
One of the leaders of the study is Jean-François Bonnefon, research director at the Centre National de la Recherche Scientifique at Toulouse and is one of three researchers working on the project. The others are Iyad Rahwan, from Massachusetts Institute of Technology and Azim Shariff, from the University of California.
In a year they have gathered 40 million pieces of data as part of an analysis involving millions of people who have taken part in the study.
Mr Bonnefon said human beings now have to make decisions about issues that they have never had to face before:
Ad
“At the moment we have not had to ask these questions.
“When we drive we do so without having decided exactly what we will do if we have an accident. We rely on the fact that we will act impulsively and do the best we can in the circumstances. We have not had to think about what we would do if we had to choose between killing the person in the passenger seat or a child crossing the road in front of us.
“These are difficult questions but if we leave these actions to a machine, we need to programme it and we have to decide how.
Ad
“It is the first time that we must think about using a machine, a car, that might choose to kill its owner in an accident when, for example, one person would be killed, rather than 10 outside the car.
“When we started looking into the subject, we discovered that there was a common consensus that the greatest number of people should be saved.
“However we found that there was an exception when a child was involved. Then it might depend on the number of people and age of the child.
“It showed that there was a huge number of factors to take into account. Would people feel that a pregnant woman should be saved rather than an elderly person? Was the age of the child relevant?
“There were so many variants that we could not rely on traditional research methods and so we created Moral Machine to get the most people responding to as many different scenarios as possible.”
Users are presented with a series of situations in which a collision involving a driverless car will kill two sets of people – it is then up to the user to judge which outcome is ‘more acceptable’.
“Sometimes the decision is easy,” says Mr Bonnefon. “For example, between five people and a cat but sometimes it is complicated.
“We now have a clear idea of the outcome but we cannot reveal our findings yet as they have to be ratified by the scientific world first. However, I can say that children are definitely priority for being saved.”
It is estimated that there will be 90% fewer accidents with driverless cars, but the researchers say we must be ready to deal with the 10% of cases that will exist.
The scientists point out that any accident caused by a driverless car will hit the headlines and could turn the public against them.
However, first the technology has to be developed to allow the cars to make these moral judgments: “It will be difficult and is another challenge for the designers. For example the car will have to be able to recognise the difference between an animal, a child and an adult, as well as having to count the number of people involved. This technology will have to be developed to react to a situation which is very unlikely to occur so there has to be the political will to include this issue in car design.”
He says it is a matter for politicians and heads of state.
“Car manufacturers do not want to have to take the responsibility so it will be left to the lawmakers to decide.
“We hope our research will help them come to the decisions which best match public opinion. It would be a shame if such an amazing invention, which can reduce fatalities on the road, were to be blocked because we cannot decide what moral decisions a car should make.”
One of the ‘moral’ decisions people taking part in the ‘serious game’ Moral Machine are faced with....should the driverless car drive on, killing a woman and three men (one of whom is a criminal) or should it swerve and instead hit two women doctors, one male doctor and a male executive? The status of people has been added to further explore moral choices.
Moral differences country by country
Moral Machine researchers found that different nationalities view these decisions in markedly different ways, which may make it difficult to come to an international agreement about the “behaviour” of machines.
Mr Bonnefon said public opinion will decide whether people buy driverless cars or not: “In Germany the government brought together a panel of philosophers and specialists in ethics to look at these problems.
“Their conclusion was that it would be immoral for a car to discriminate against a person on age grounds. However our studies show that public opinion differs.”
The German model – which focuses on the first iteration of self-driving cars in which control can be switched to a human driver – insists that, in the case of a collision in which the computer is in control, it must make no discrimination on the basis of age, gender, race, physical attributes or anything else of any potential accident victim.
All vehicles should also have an aviation-style “Black Box” that continuously records events, including whether a human or the computer is in control at any given time.
In France, driverless buses have already transported passengers in Lyon and Paris, while car manufacturer PSA is planning to have fully driverless cars on the road in 2020. Four of its self-driving cars drove 360 miles between Paris and Bordeaux in October 2015.
Cars with a self-drive option are expected on the streets of France this year.
