BlackBerry Cylance AI-based anti-virus 'tricked' into passing malware off as safe

Researchers were able to trick a popular AI system into thinking ten of the most dangerous forms of malware were good.

Researchers in Australia claim to have tricked a popular anti-virus security system, supposedly based on artificial intelligence (AI) rather than signatures, into thinking malware isn't harmful.

Using a "global bypass method", specialists at Skylight Cyber were able to get BlackBerry Cylance's PROTECT system to identify malware as "goodware".

They continued: "AI applications in security are clear and potentially useful. However AI-based products offer a new and unique attack surface.

"Namely, if you could truly understand how a certain model works, and the type of features it uses to reach a decision, you would have the potential to fool it consistently, creating a universal bypass."

If you could truly understand how a certain model works, and the type of features it uses to reach a decision, you would have the potential to fool it consistently

According to the researchers, they identified "a peculiar bias towards a specific game" after conducting an analysis of the system.

The researchers continued: "Combining an analysis of the feature extraction process, its heavy reliance on strings, and its strong bias for this specific game, we are capable of crafting a simple and rather amusing bypass."

The added that by appending a selected list of strings to a malicious file, they could change its score significantly to avoid detection. "This method proved successful for 100 per cent of the top 10 Malware for May 2019, and close to 90 per cent for a larger sample of 384 malware."

We are always amused to see the shock on people's faces when you tell them that the new security toy they spent millions of dollars buying and integrating can be bypassed

To test their method, the researchers uploaded a list of the top ten malware (published by the Center for Internet Security) and called the results "staggering".

The results show that negative scores were turned to positive, meaning some of the most dangerous forms of malware were able to escape detection.

Discussing the results, the researchers said: "Almost all of these samples have changed from the most evil file on the planet, to your friendly neighborhood file. Again, the only treatment applied to these files, is the addition of the ‘special sauce' as a simple concatenation."

After downloading a list of 384 malicious files and running them through the test, the researchers generated the following results:

• Average score before secret sauce: -920
• Average score after secret Sauce: 630
• Average delta: 1550 (out of a maximum of 2000)
• Percentage of files bypassing detection: 83.59%

They concluded: "We are always amused to see the shock on people's faces when you tell them that the new security toy they spent millions of dollars buying and integrating can be bypassed.

"The same goes for new silver bullets, like AI based security. We are anything but surprised with the results, and we are confident that the same type of process can be applied to other pure AI vendors to achieve similar results."