How artificial intelligence is changing security

The city of Nice could soon be using security cameras so ‘intelligent’ that they can identify members of the public who look afraid or angry.

There are plans to trial a system in the city’s trams that is capable of automatically recognising unusual emotions, such as fear or anger, on the faces of passengers and flagging it up, a mairie staff member said.

Nice already considers itself one of France’s ‘smartest’ cities.

As Connexion reported in August, the city’s municipal police claimed a 100% success rate with facial recognition tests during the city’s carnival.

The mairie told Connexion when we visited that it hopes its report about this to the government and MPs will spur on laws allowing facial recognition to be used daily [since then however the initiative hit a bump in the road with national data security agency Cnil saying the report was not detailed enough for final conclusions to be drawn].

Mayor Christian Estrosi believes current personal data laws which prevent it, dating from 1978, are obsolete.

Nice has not waited for such a change, however, to make massive use of video surveillance. Up to 20 people work 24 hours a day in its Centre de Supervision Urbaine (CSU), the largest of any French municipal police.

They monitor screens that show images from 2,700 cameras, take calls, and process requests from other authorities to use images. The cameras are in strategic points around the commune, as well as at its 153 schools and at tram stops and inside trams.

CSU head Grégory Pezet said the centre was created in 2008 by Mr Estrosi. “When he arrived, we had 300 cameras, and there will be 3,000 by the end of the year.” The cameras are 4k resolution, giving high quality images.

“We can zoom in to see a car numberplate at 600 metres,” Mr Pezet said.

Some present a fixed view, while others work in 360 degrees. “The operator uses a joystick to zoom in as they wish.” Quadra cameras provide images in 360 degrees simultaneously from four lenses arranged in a dome.

Operators are in contact with police on the ground and also keep watch on serious traffic situations or extreme weather conditions, he said.

“At the start of the year we had waves crashing up on the Promenade des Anglais and we were able to keep an eye on how things evolved so as to close off the road if necessary.”

The Police Nationale, gendarmerie and pompiers also have screens which can access their cameras. “If there’s a big fire or a gas explosion, for example, they can use them to adapt what teams and equipment need to be sent.”

Sometimes cameras are used for vidéoverbalisation – sending a fine to a person’s home, based on behaviour picked up – though Mr Pezet said this was not the main reason for them.

“We make use of it for recurrent problems, like double parking or parking on cycle lanes and pedestrian crossings or taxi ranks. We get a lot of accidents because bad parking has caused jams,” he said.

Every school has cameras monitored by dedicated officials, especially in the mornings, lunchtimes and afternoons when the children are coming and going. “Our mayor wants to emphasise making schools safe,” Mr Pezet said.

The cameras are used for big events, such as the carnival or jazz festival.

At such times, operators check for any suspicious behaviour. The event’s overall security is managed from a command centre at the CSU, with representatives from the prefecture, event organisers and the emergency services, including a police commissioner.

One special piece of kit is a thermal camera which picks up unusual movements of heat – such as people’s bodies entering unauthorised areas.

Nice also prioritises using artificial intelligence and is a “real laboratory” for trying such technology, Mr Pezet said. “We have software that allows us to look for specified objects: for example, a person who has a white shirt, black trousers and is carrying a case. We can do the same for vehicles and can look for a white van, a blue car, etc.”

This is already legal because it does not concern a specific individual.

Mr Pezet said they receive many requests for images from the Police Nationale and gendarmerie to help in inquiries. “In 2018, 1,700 cases were solved thanks to camera images and I think there’ll be 2,000 this year.”

Judges and public prosecutors make increasingly frequent demands, he said. “An image doesn’t lie. It gives clear information about a crime and can be an indispensable piece of evidence.” However, the process of extracting images is regulated and can only be done on request from judicial police

Another common use is un flag – flagrant délit – around 600 times a year.

“The operators spot a person acting suspiciously, follow them and – bam! – they commit a crime. A team of police is sent and we provide images for the justice system.

“At other times, the Police Nationale contact us and say, for example, someone matching such-and-such a description has just carried out a mugging. I put the operators on it and they often manage to pick them up.

“Today we cannot do without the cameras for security. But it can’t resolve everything, there’s no such thing as zero risk.” Mr Pezet said the camera system was unable to stop the 2016 Fête Nationale terrorist attack because the truck the attacker used was not seen doing anything unusual beforehand.

The best solution is using intelligence to prevent such crimes before they happen, he said.

Mr Pezet said in the early 2000s many people had misgivings about cameras, due to privacy, but that has changed.

“The tendency is now the opposite.Each month we have residents’ groups and individuals asking us to install them because people feel reassured to have one in their street.” Several times a year, a committee considers requests.

“We install 200-300 a year, but can’t have one every 10 metres.”

Due to privacy issues, the surveillance system blurs out windows in the streets.

Some other councils also have innovative techniques. Saint-Etienne has been testing cameras with microphones that can pick up unusual noises, such as the crash of a road accident or gunfire, sending an automatic alert.

Others, such as Pontoise in Ile-de-France, have cameras with speakers – which Nice is considering. “In meetings we’ve raised having them on some cameras, to tell people, for example, that they must leave a public garden that’s about to close,” Mr Pezet said.

System could even tell apart identical twins

The City of Nice council told Connexion they had a 100% success rate with the facial recognition test at the carnival in February, meaning that their cameras located all of the test subjects and identified no one else.

The software, by Monegasque firm Confidentia (see below), works by marking out the person being sought with a coloured box on the image.

There were three days of tests, trying to find eight volunteers from supplied images. Scenarios included a child lost in a crowd, an elderly person with Alzheimer’s who is lost, and even identical twins to see if the system could differentiate between them.

It did – it did not flag the ‘wrong’ twin, but then it did identify the one it had been instructed to find.

The system even managed to ident-ify a person who had supplied a photo that was more than 30 years old – and two people wearing disguises such as a fake moustache, wig, glasses or cap.

It was also tested with a supplied photo that was deliberately of very poor quality: low resolution and lacking sharpness.

A single photo is enough for the system to spot someone, even if the face is not looking straight at the camera. The photo does not have to be sharp, Nice mairie found.

The system works by matching characteristics of the supplied image with people in view of the camera, and it can ignore all irrelevant factors, such as changes in hair or clothing.

Images are saved in servers for 10 days before being destroyed automatically.

Mr Pezet said: “When investigators from the Police Nationale or gendarm-erie ask for images, we can’t supply older ones.”

Ten days already represents 1.5 petabytes – one and a half billion megabytes, equivalent to the storage on 213,000 movie DVDs.

The test in February was authorised by the Interior Ministry and the Cnil agency and the city reported the results to them and to MPs.

For the system to be used in real life, it would need a new law to be voted through by Parliament.

‘Intelligent’ technology has many uses, says firm that supplied Nice

The firm behind the software that Nice used in its facial recognition trial, Confidentia, was founded in Monaco in 2015 by French private funders.

It specialises in high-tech security.

Apart from facial recognition, it also supplies a mobile phone protected from all forms of snooping – eg. for governments and company boards – and an ‘unhackable’ phone system for private banks and their clients.

Chief operating officer Stéphan Louppe said: “Mobile phones are more and more becoming the entry point for attacks [on data] and are often connected into professional IT systems.

“We saw that video surveillance was being more and more used for security, as well as access to events, and so it’s a big market with many potential uses.

“Nice has a huge number of cameras, but just having a lot of cameras isn’t enough. You need people or applications to process all the videos. What we do with facial recognition is automated and is more reliable and accurate than a human. Plus the police do not want their operators to spend all their time just watching cameras, but would rather they react to what is detected.”

The system uses artificial intelligence, video and image processing and deep learning. “It detects a face in an image, and a video is a series of images, many of them per second. It knows what a face is and for each one detected it calculates mathematical vectors, which are the face’s biometric footprint, which it compares to reference images [ie. images of searched-for people].”

He said the ‘vectors’ calculation is more sophisticated than simple measurements such as the distance between the eyes and is not thrown off by sunglasses, a helmet or shaving a beard. It has a detection rate of 95-99%, depending on the quality of the image supplied.

“It can work in real time – it takes less than 0.2 seconds to calculate and compare all faces in the field of view, whether that’s two or a marathon crowd. You can also replay sequences and use it after the event as part of an investigation.” As a test of the latter, Nice used footage of the city’s half-marathon and the system was able to pick out mayor Christian Estrosi running.

Mr Louppe said facial recognition can be combined with body and object recognition. “If you know you are looking for someone with this face wearing a black jacket – if you give the system that information, it may improve the result.”

Other potential uses include managing a blacklist – flagging up if certain people try to enter a space such as a private villa or yacht – or whitelist, ie. detecting people not on the list who try to come in.

It can also be used for personal identification for accessing a building or a computer application.

Football fans could look at a camera and be allowed through a gate.

Firms can use ‘intelligent’ cameras to compile statistics – a mall can find out how many people visit, their ages and gender, and what parts of a shop attract their attention first.

Mr Louppe said laws on facial recognition are more flexible in Spain and the UK (even more so in South America).

However, a recent Essex University study said the system tested by London’s Metropolitan Police since 2016 identified the wrong people 81% of the time.

Confidentia believes its system, now used in several cities around the world, is more reliable. “It is strong and accurate and really pushes the limits,” Mr Louppe said. “Nice is really pushing for it and the aim of the tests was to show it works and meets objectives in a real operational environment.

“We have built in data security features, like the option to only record faces of a ‘person of interest’. We also have a mode where that person’s image is visible but other people are blurred. All the bio-metric data is stored anonymously. Now we’re waiting for the law to change.”

Confidentia has 10 staff in Monaco, plus staff in Paris, Italy and Spain, and research and development in Israel.

More on this topic: French debate on facial recognition