Unlike traditional computer algorithms, AI doesn’t follow predefined rules. It analyzes many complex inputs, producing powerful statistical models that can recognize images, interpret human speech, and even "read" text to extract key concepts. This ability to carry out small but complex tasks makes it a useful tool for attackers who need to do this often and at scale.
Reconnaissance is one of the first stages in an attack and one area where AI could help attackers hone their efficiency. During this phase, the would-be intruder scours the public domain for information about the target. In early cyberattacks, this mainly involved network scanning. These days, there is so much information available through social media sites, company reports and news articles that attackers are drowning in targeted intelligence.
AI promises to be the perfect tool for collecting, collating and analyzing this data. In 2017, researchers at Black Hat presented a system for mining data about organizations and executives and using machine learning to process it for business email compromise (BEC) attacks. The result was an automated tool for creating convincing emails with spoofed header fields.
Reconnaissance AI can also pick up valuable information about how people write. A year before the Black Hat presentation, cybersecurity company ZeroFOX wrote a machine learning-based software program that analyzed Twitter posts from targeted individuals along with a broader corpus of general tweets. It used this information to automatically create realistic-looking tweets targeting high-value users that could be seeded with a malicious URL.
A new era of fake
Mechanically gathering and processing large amounts of data for use in automated attacks makes it easier for hackers to scale up their operations.
“Attackers need highly repeatable models that allow them to carry out the same attacks time and time again on different networks,” explains Charl van der Walt, Head of Security Research at Orange Cyberdefense. “Not every attack will be successful, but the more attempts they make, the increased probability of effective penetration of a network. It’s a numbers game.”
AI also makes these attacks more convincing. The days of poorly-worded phishing attacks littered with misspellings may be drawing to a close.
AI is getting better at replicating people, not just in the written word but also in images and videos. Deepfakes use generative adversarial networks (GAN), which are machine learning-based neural networks that compete against each other to produce the most convincing outcome. One network tries to create an artificial artifact while the other tries to spot flaws in it. They create a positive feedback loop, each trying to better the other until a convincing asset emerges. Intelligence services have already used deepfakes in social engineering attacks on LinkedIn to build networks of contacts.
While hobbyists around the world have stunned us with video-based deepfakes that seem to show comedian Bill Hader morphing into Tom Cruise, a more immediate danger to companies may come from voice cloning. This technology applies similar effects to video, enabling people to mimic others’ voices in real time.
Voice deepfakes create plenty of opportunities for social engineering via phone. Since 2019, organizations have seen multiple cases of attempted voice deepfake fraud as attackers train neural networks to impersonate high-value targets using footage from YouTube and elsewhere.
Software-based hackers
Attackers are making advances in other malicious AI use cases, too. In 2016, DARPA held a Cyber Grand Challenge that pitted teams with AI-powered software algorithms against each other. With impressive results, each attempted to either hack or defend electronic networks in a game of automated cat and mouse.
Researchers have been busy creating algorithms that use machine learning to test cybersecurity defenses and, in some cases, bypass them altogether. At the same Black Hat conference that saw the BEC automation attack, researchers also presented AVPASS, a tool that used AI to work out how antivirus programs detected malware in Android applications and then disguised software to circumvent them – with a zero-percent detection rate.
Security firm Bishop Fox also demonstrated DeepHack, a tool that uses machine learning for automated web application penetration testing. Others have created GAN-based password-guessing software. Whereas traditional systems use brute-force techniques to iterate through millions of passwords, PassGAN uses AI to extrapolate from previously-leaked passwords, using probability scores to come up with likely variations. The result is a system that could make it easier for attackers to access supposedly secure accounts.
Van der Walt expects AI attacks to grow in sophistication over time, pervading even more parts of the attack chain. “The types of AI-based attacks we expect to see include ‘DeepPhishing’ (using deep learning to bypass AI-based phishing detection), fooling deep learning-based image recognition, web application attacks, and bug hunting in libraries,” he warns, adding that these techniques are all at early stages of development.
Other possible criminal uses for AI include detection evasion. Software might watch the environment around it after it has infected a target, modifying its behavior to blend in with existing traffic. It could also make decisions about how it attempts to spread to the rest of the network and even about which data to target and exfiltrate.
Do not despair
We must not panic over the malicious possibilities of AI. They exist, but we are not defenseless. We must consider possible mitigations and look at the potential upsides of AI from a cybersecurity perspective. As Europol outlines in its November 2020 report on AI and cybersecurity, we can promote best governance practices around this technology that will help to keep things secure. For example, we could encourage those publishing new AI research to include information that would help detect or otherwise mitigate the weaponization of their work. There is already counter research that can help automatically detect deepfakes by spotting anomalies in GAN-generated images and video.
We can also use AI ourselves as another layer of defense. Financial services companies already use it for fraud detection, while numerous companies are now using machine learning to detect abnormalities in everything from software to emails and even user behavior.
As with most enabling technologies, AI can be a weapon or a tool. For every attacker who chooses to use it irresponsibly, there are plenty of engineers creating powerful cybersecurity tools that could use AI to turn the tables on attackers.
The Orange Cyberdefense Security Navigator 2021 report has been released. It includes first-hand information from our 17 global SOCs and CyberSOCs, the Epidemiology Labs and World Watch as well as expert reports and technology reviews on topics like videoconferencing solutions and the cybercrime ecosystem. Download the report to find out what the most disrupting events in 2020 were and how that projects into the future.