An AI broke Snowflake's code. Then another AI agent exploited it

Security

Don't worry, this one was via a bug bounty program

An AI broke Snowflake’s code; then another AI, an attack agent, autonomously found the bug, exploited it, and extracted credentials without human intervention.

Luckily, this wasn’t yet another case of rogue AI agents doing evil things. It was a sanctioned bug hunt, conducted through Snowflake’s HackerOne vulnerability disclosure program, and Snowflake fixed the flaw the same day Wiz reported it and rotated the affected credentials the following day.

Wiz’s red agent, an AI-powered autonomous attacker designed for offensive security, found the GitHub Actions workflow flaw during a routine scan of public repositories on June 23. The script injection vulnerability existed in snowflakedb/snowflake-connector-net, and it allowed an unauthenticated user to execute arbitrary commands within a GitHub Actions runner by opening a GitHub issue with a specially crafted title.

And it turned out an AI had inadvertently injected the bug into the code five days earlier.

GitHub Copilot Autofix, an AI coding assistant, co-authored the commit on June 18, and it introduced a script injection bug in run: blocks by removing the repository’s existing sanitized input pattern and replacing it with direct string expansion in a shell script.

“We crafted an issue title that, after template expansion, breaks out of the echo string and exfiltrates the Jira credentials via an out-of-band callback,” Wiz’s head of threat exposure Gal Nagli said in a Monday blog. 

These credentials gave Wiz read access to Snowflake’s engineering, security compliance, and bug bounty tracking projects.

Wiz reported the workflow vulnerability to the cloud data platform on June 23, and Snowflake patched it the same day. It also revoked and rotated the Jira token, and confirmed, via audit logs, that Wiz was the only third-party to access the endpoint during the five-day exposure window. 

The disclosure “was immediately investigated and remediated, and our investigation found no evidence of unauthorized access,” a Snowflake spokesperson told The Register. “We are working together with Wiz to share these learnings with the broader industry to encourage widespread adoption of these security best practices.”

Wiz, for its part, deleted all of the data it accessed during the vulnerability research and proof-of-concept exploit testing, and told us that this incident proves human code review isn’t sufficient to quickly detect vulnerabilities -  especially as developers increasingly use AI.

“This incident highlights a rapidly emerging reality in software development: how AI coding assistants can inadvertently introduce workflow injection vulnerabilities, and how automated AI agents can rapidly surface them in the wild,” Nagli wrote.

Of course, the Google-owned biz has a vested interest in saying this. But this doesn’t make it not true.®