4 uses of machine learning in cybersecurity today

We take a look at some of the key ways machine learning is being used today to detect malware, analyse data and spot suspicious user behaviour.

By Andy McCue

Sun 2 Jul 2017 @ 12:08

Machine learning is rapidly rising up the cybersecurity agenda in the face of the increasing volume and sophistication of security incidents, combined with the ongoing shortage of security and data analysis skills to deal with them.

Some 80 per cent of security incidents are reportedly going undetected, while 42 per cent of cybersecurity professionals admit their organisation ignores a significant number of alerts because they can't keep up with the volume. The security skills shortage is exacerbating this picture. Research by ISC2 says the global security industry is short of a million professionals and that this will grow to 1.8 million within five years.

The security industry is turning to machine learning to help tackle this challenge. Analyst Gartner forecasts a quarter of security products used for detection will use some form of machine learning by 2018, while ABI Research predicts machine learning in cybersecurity will boost Big Data, intelligence and analytics spending to $96bn by 2021.

“We are in the midst of an artificial intelligence (AI) security revolution,” says ABI Research analyst Dimitrios Pavlakis. “This will drive machine learning solutions to soon emerge as the new norm beyond security information and event management (SIEM) and ultimately displace a large portion of traditional AV, heuristics, and signature-based systems within the next five years.”

It's worth making the distinction between machine learning and AI in this context. Machine learning is not AI. Machine learning still requires some human intervention and engineering but the technology uses algorithms and predictive models to sift through and monitor the security noise in real-time and flag up things that might need investigating by the organisation's security team.

Here are four uses of machine learning in cybersecurity today.

1. Malware detection

Hunting for malware has become increasingly challenging for traditional anti-malware security technologies and for over-stretched security teams. The figures reveal the scale of the challenge. According to the AV-TEST institute there are nearly 400,000 new malware variations every day. However, machine learning is now replacing or at least complementing signature-based and heuristic malware detection. Machine learning algorithms can rapidly analyse file features and behaviour and flag up those that could be malicious for further investigation by a data analyst. However, there is still some human resource required to set those parameters and variables that the machine-learning algorithm uses to analyse file features and behaviour.

2. Dynamic risk and threat analysis

Machine learning can help make some sense of the vast quantities of data through continuous real-time monitoring and a sophisticated Big Data analytics approach. This can deliver human operators accurate, actionable intelligence on real threats.

3. Monitor user behaviour and the insider threat

The advanced analytical capabilities of machine learning are increasingly being used in so-called user and entity behavioural analytics (UEBA) security technologies. In this context the machine-learning algorithm uses historical user behaviour data, such as login times, locations and devices to build a picture of 'normal' activity. It is then able to analyse network activity in real-time to rapidly detect anomalies that could signal compromised user accounts or malicious employee activity such as privilege abuse or data theft. Analyst Markets and Markets says the UEBA market will grow from $131.7m in 2016 to $908.3m by 2021.

4. Deep learning

This is still in its relative infancy in terms of its application in cybersecurity but some predict it will be the next evolution in advanced security machine learning. Using a neural network model based on the way the human brain learns to recognise things, deep learning enables machine-learning algorithms to learn without human intervention and the manual setting of parameters. Early tests show that this can be more effective than current machine-learning techniques in detecting zero-day malware and advanced threats – one test of a deep learning technology detected more than 99.9 per cent of malicious code.