Melissa Heikkilä reporting for MIT Technology Review: “We are hurtling toward a glitchy, spammy, scammy, AI-powered internet”

The advent of cheap, universally accessible AI (using large language models such as GPT) might prove to become a security nightmare:

I have some bad news: It’s stupidly easy, it requires no programming skills, and there are no known fixes. […] Let me walk you through how that works. First, an attacker hides a malicious prompt in a message in an email that an AI-powered virtual assistant opens. The attacker’s prompt asks the virtual assistant to send the attacker the victim’s contact list or emails, or to spread the attack to every person in the recipient’s contact list. Unlike the spam and scam emails of today, where people have to be tricked into clicking on links, these new kinds of attacks will be invisible to the human eye and automated.

The attack vector is similar to SQL injections, whereby an attacker inserts SQL strings into data entry fields on, for example, websites that then get executed (if not filtered out). In the case of AI prompt injections, the challenge lies in the “there are no known fixes” part – not saying that we won’t come up with smart ways to filter these out, but it is not trivial (other than with SQL injections which require a rather simple escaping of data).

Add to this the fact that AI allows you to (as WIRED just pointed out) “thanks to large language models, a single scammer can run hundreds or thousands of cons in parallel, night and day, in every language under the sun” and you truly have a security nightmare at your hands.

As Heikkilä summarizes:

It’s a shitstorm we are not even remotely prepared for.

😭