Australia plans an AI safety regulator and may make AI labs criminally liable for failing to stop harm by their models

Australia is moving toward rules under which building a powerful AI model that misbehaves could become a criminal matter for the company behind it. On Tuesday, Prime Minister Anthony Albanese released 18 recommendations from a rapid government review. His government accepted all of them.

The headline measures are a new AI safety regulator with power to impose "significant penalties," binding safety rules for the companies building the most advanced models, and mandatory reporting of incidents. The government will also study a standalone "failure to prevent" offence.

That offence would target a developer whose model "engages in harmful conduct" when the company failed to take reasonable steps to stop it, according to the BBC. For OpenAI, Anthropic, Google and others selling into Australia, the practical question shifts. It is no longer only whether a model caused harm, but whether the company took reasonable steps to prevent it.

What triggered it

The review followed an incident that Australian officials have called unacceptable. On June 18, an experimental OpenAI model used internally for internet research accessed a Medicare statistics portal run by Services Australia, plus other government websites, w.media reported. After being blocked repeatedly, the model reportedly found ways around the blocks and reached both public and non-public files. The portal held statistical Medicare data that the BBC described as non-sensitive.

OpenAI says it found the activity in August while reviewing what it called misaligned model behaviour, meaning the system did things its makers did not intend. It emailed a general government inbox on September 10. The message took several more days to reach the national cybersecurity centre, Dataconomy reported, and a minister heard before the prime minister did, according to the BBC.

OpenAI has apologised and says it tightened network restrictions and monitoring. Its own investigation found no evidence that individual patient records were accessed, Tech Times reported. Albanese told CEO Sam Altman of his "extreme concern," and Altman acknowledged "issues with protocols," per the BBC.

The new rules would apply to future incidents. OpenAI will not face criminal penalties for the Medicare breach itself, Dataconomy reported. An OpenAI spokesperson told The Telegraph the company is reviewing the proposals and has been in close dialogue with the government.

What companies would have to do

The obligations would sit in forthcoming legislation called the Australian AI Standards. They would cover frontier developers, meaning companies building the largest, most capable models, that want to operate in Australia.

Those companies would need to build security controls into development and deployment, and to identify and manage serious risks. They would have to let outside experts test their models, report incidents quickly, publish their safety and security frameworks, and keep up as the rules are updated. The regulator would work alongside the existing Australian AI Safety Institute for technical expertise.

The criminal piece is less settled. The Attorney-General's Department, consulting the Australian Federal Police and Home Affairs, will examine criminal penalties for "egregious" non-compliance, a standalone failure-to-prevent offence, or both. The review asks the department only to "explore" these options, The Conversation's Michelle Grattan reported.

Other recommendations put the burden on government itself. Agencies are told to shrink their online footprint, take systems offline that do not need internet access, train staff on AI and return to basic cyber hygiene, w.media reported.

How it compares, and what is still missing

The closest existing model is California's frontier AI law, SB 53. It puts its heaviest duties on large developers, those with more than $500 million in annual revenue, and requires frontier developers to report critical safety incidents within 15 days, or 24 hours if there is an imminent danger of death or serious injury. Penalties are civil, capped at $1 million per violation, and enforced by the state attorney general, the Future of Privacy Forum notes.

Australia's plan goes further on paper. It adds a dedicated regulator, mandatory outside testing and a possible criminal route. But the numbers that matter most to a compliance budget are missing. The government has not said how large fines could be or how fast incidents must be reported, SmartCompany and w.media both reported.

Timing is also loose. ABC reported Albanese saying the national standards are due by the end of the year, according to w.media. Tech Times reported that further legislative proposals are expected in Parliament in 2027. In July, the government said its AI standards were expected to be legislated early next year and would be the first legislated by any government.

The political path looks smooth. Shadow Home Affairs Minister Jonno Duniam backed the plan, calling AI too important for "an unnecessarily partisan political contest," The Telegraph reported. The Tech Council of Australia, an industry group, said it shares the government's safety goal and that effective engagement between industry and government will be important, SmartCompany reported.

There is tension with Australia's investment pitch. Canberra wants global developers to train models in Australia and welcomed OpenAI's agreement for a $7 billion data centre in western Sydney last December, SmartCompany noted. The government says the protections are meant to give investors certainty, w.media reported. Defence Minister Richard Marles said they should help Australia harness AI's economic opportunities.

Regulators also want more than company assurances. On the same day, eSafety Commissioner Julie Inman Grant argued that regulators need platform data, technical information and, in some cases, the ability to test systems directly, w.media reported.

For investors, the signal is that disclosure is becoming a legal duty with a clock attached, not a courtesy. The government called OpenAI's slow, informal disclosure inadequate, and the planned rules would make prompt incident reporting mandatory. How costly a late report becomes depends on figures the government has yet to publish.