IBM warns of AI-powered audio-jacking in live calls

Getting your Trinity Audio player ready...

Generative artificial intelligence (AI) has made it surprisingly easy to hijack and manipulate live conversations, relaying malicious information to either party, IBM (NASDAQ: IBM) researchers say.

The researchers modified the context of a live audio call between two parties using large language models (LLMs), voice cloning, and text-to-speech technologies.

In their exploration, the researchers intercepted a live audio call and listened for the words “bank account.” Once any party mentioned a bank account, the LLM would clone their voice, cut them off from the conversation, and replace it with a fake account.

The researchers noted that they decided against using the cloned voice for an entire conversation as it would be easy to notice. The study further shows that the two targets whose live audio call was intercepted had no idea that their connection had been compromised.

“Alarmingly, it was fairly easy to construct this highly intrusive capability, creating a significant concern about its use by an attacker driven by monetary incentives and limited to no lawful boundary,” the researchers said.

The attack focused on manipulating the bank account information of its targets, but according to IBM, there’s no limit to what an audio-jacking attack can achieve. Attackers could employ this tactic to modify medical information such as allergies or blood type to devastating effect. It could even instruct a pilot to reroute a plane on a broader scale.

“Building this PoC was surprisingly and scarily easy,” the researchers say. Before AI, training a system to learn the semantics of a conversation and modify even one sentence would take a dedicated team of researchers.

However, in the AI era, “LLMs make parsing and understanding the conversation extremely easy.”

The developers even accounted for expected pauses as the LLM system formulated the sentence with the cloned voice. Aware that pausing could arouse suspicion, they bought the system a few seconds with bridging phrases like “Sure, just give me a second to pull it up.”

With AI presenting novel threats to consumers, the need for cutting-edge cybersecurity solutions has never been greater. Certihash has emerged best-in-class, leveraging the Bitcoin blockchain to offer robust and pragmatic cybersecurity solutions equipped to deal with the ever-evolving threats.

Its flagship product, Sentinel Node, cuts down the threat detection time from an industry average of over 200 days to seconds. Developed in partnership with IBM, Sentinel Node allows victims to respond immediately, potentially saving millions of dollars.

In order for artificial intelligence (AI) to work right within the law and thrive in the face of growing challenges, it needs to integrate an enterprise blockchain system that ensures data input quality and ownership—allowing it to keep data safe while also guaranteeing the immutability of data. Check out CoinGeek’s coverage on this emerging tech to learn more why Enterprise blockchain will be the backbone of AI.

Watch: Blockchain can bring accountability to AI