Researchers develop way to turn commands into voice-activated AI software

New attacks against supposedly intelligent assistants demonstrated by US-Chinese researchers

A team of computer scientists have found a new way to trick voice-activated systems into surreptitiously executing commands embedded in popular songs.

In the project, researchers in the US and China teamed up with technology company IBM to explore potential security threats to popular virtual assistants, such as Apple's Siri, Amazon Alexa and Google Assistant.

The researchers explain how deep learning is an integral part of these systems. But, just like computer-vision technologies, voice-based smart assistants can also fall victim to "adversarial perturbations".

These adversarial perturbations can be used to trick machine-learning and artificial intelligence systems into identifying the wrong objects, and hence, following the wrong instructions. Cyber crooks can use them to fool popular smart assistants into, for example, making purchases the owners did not intend to make.

"In this paper, we not only demonstrate such attacks can happen in reality, but also show that the attacks can be systematically conducted," said the researchers.

To do this, they embedded voice commands into a song called 'CommandSong'. "In this way, the song carrying the command can spread through radio, TV or any media player installed in the portable devices like smartphones, potentially impacting millions of users in long distance," they explained.

The researchers ran into some problems, but they bypassed them. "In particular, we overcame two major challenges: minimising the revision of a song in the process of embedding commands, and letting the CommandSong spread through the air without losing the voice command," they said.

Their research indicates that it's possible to "craft random songs to any commands and the modify is extremely difficult to be noticed". The researchers achieved a 94 per cent success rate.

Ten authors took part in the report, including Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Carl A. Gunter and Xiaofeng Wang.

The research is not the first to highlight potential security shortcomings in supposedly smart assistants. In September, a group of researchers demonstrated how the devices could carry out commands based on instructions that are inaudible to the human ear.