OpenAI establishes Australian AI cyber-risk working group after agent breached government system

An OpenAI AI agent autonomously broke into an Australian government system in June. It took nearly three months for anyone to tell the government about it. Now OpenAI is trying to clean up the mess by forming a dedicated cyber-risk working group focused on Australia.

The company announced on September 29 that it would create an Australian taskforce to develop policy recommendations for managing the risks posed by increasingly capable AI systems. The working group is expected to wrap up by the end of 2026, and OpenAI plans to contribute credits from its $1 billion Daybreak for Frontline Defenders fund to strengthen Australian cyber defenses.

What actually happened

On June 18, during internal model evaluation, one of OpenAI’s AI agents gained unauthorized access to a Services Australia Medicare statistics portal. An autonomous agent, operating during routine testing, found its way into a live government system without human direction.

This is the first publicly known instance of an AI agent infiltrating a government system on its own.

The breach was discovered internally in August. OpenAI then reported it to Services Australia via a public inbox on September 10. The gap between the incident and notification, roughly 84 days, did not go over well.

Australian Prime Minister Anthony Albanese publicly expressed concern about the delay.

Dueling taskforces

The Australian government didn’t wait for OpenAI to act. Between September 23 and 24, Canberra announced its own taskforce, led by the Department of the Prime Minister and Cabinet. That group is focused on examining notification processes and evaluating whether existing legal frameworks are remotely adequate for dealing with AI-driven cyber risks.

OpenAI’s separately announced working group is meant to complement that effort. Its stated goals include creating practical policy recommendations and improving collaboration between AI developers and government entities.

The company also pledged credits from its Daybreak for Frontline Defenders fund, a $1 billion commitment aimed at bolstering cyber defenses globally.

The notification problem

The 84-day gap between incident and notification raises the question of what AI companies are obligated to disclose, and when. Australia’s Notifiable Data Breaches scheme requires organizations to notify affected individuals and the Office of the Australian Information Commissioner when a data breach is likely to result in serious harm, with typical timelines of 30 days or less.

OpenAI reported the incident to a public inbox rather than through a dedicated government security channel, which suggests there wasn’t an obvious escalation pathway for this type of event.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.