With new AI systems comes new AI vulnerabilities, and a big one was just discovered. It’s a flaw in Microsoft’s method of allowing agents to interact with websites on your behalf.

Microsoft calls this technique NLWeb, which is a kind of HTML for AI agents. The company unveiled this at its Build conference this spring, and has since leaned into that vision with an experimental Copilot Mode for its Edge browser. (Microsoft hasn’t confirmed whether it uses NLWeb for this.)

Researcher Aonan Guan, however, has discovered a vulnerability in NLWeb: a path traversal bug that lets any remote user read sensitive files like system configurations and cloud credentials via a malformed URL.

In a Medium post, Guan showed how he was able to download a list of the system passwords along with Google Gemini and OpenAI keys. This would let an attacker run additional server-dependent AI applications “for free,” without being charged by OpenAI.

According to Guan, Microsoft’s Security Response Center pushed a patch to the GitHub repository in June, confirming the problem was fixed. Microsoft hasn’t issued an official patch report. Users, however, don’t need to take any actions.

It’s fair to say that AI development has proceeded at breakneck speed. But, as Guan points out, the line between chatting with an AI and issuing it commands can blur.

“The very nature of NLWeb is to interpret natural language,” Guan said. “This blurs the line between user input and system commands. Future attack vectors could involve crafting sentences that, when parsed by an agent, translate into malicious file paths or actions.”

We’ve already seen ChatGPT interactions leak out into Google’s search results. (ChatGPT has now reportedly turned off the flag that makes ChatGPT chats discoverable.) As Guan (and The Verge, which reported the story) note, leaks of such magnitude in an AI agent can be catastrophic for all involved.

Author: Mark Hachman, Senior Editor, PCWorld