Trump may be forced to reveal secret rules feds use for AI safety testing

Trump’s secret reviews of frontier AI models may hide corruption, lawsuit says.

Four federal agencies have been sued amid calls to release information about the secret framework that the Trump administration uses to conduct safety reviews of frontier AI models prior to release.

In a Wednesday press release announcing the lawsuit, a nonpartisan nonprofit called Protect Democracy alleged that “almost no details” have been released to the public or Congress. To everyone except a few vague “trusted partners,” it remains unclear what the government’s review process looks like, which companies are involved in constructing the framework, or what legal authority Trump officials have to conduct the reviews.

“Neither the identities of those entities nor the criteria by which they were selected have been made public,” Protect Democracy said.

That’s a problem, Protect Democracy alleged in a statement to Ars, since decisions about “which AI models are approved and released may be the most important policy question of this White House.” And it appears that at least one AI firm, OpenAI, has “negotiated a private agreement with the Federal Government to limit distribution of its cutting-edge AI models to government-vetted partners.”

The group’s lawsuit is seeking an order requiring officials to produce all information sought by September 30. That includes “unclassified procedural and contractual architecture: the framework’s text, the terms of participation, the identity of participants, and the process and criteria by which access to frontier models is granted or withheld,” the complaint said. They also want an injunction restraining officials from improperly withholding records that aren’t classified.

“The executive branch is now choosing which companies can release their products and which customers get access to a technology that could shape the future of not only American industry and national security, but economies and governments abroad as well,” Protect Democracy’s release said. “All with no oversight from Congress, the public, or tech experts outside of the executive branch.”

Secret rules allegedly hide corruption

According to Protect Democracy, the framework could potentially be “corrupt,” with officials from the Office of the National Cyber Director, the Office of Science and Technology Policy, the Treasury Department, and the Commerce Department potentially favoring rapid deployments for AI firms that Trump likes. And on the flip side, the public has already seen Trump retaliate against an AI firm for being too woke, Protect Democracy noted, pointing to a judge’s recent ruling that it was illegal for Trump to blacklist Anthropic.

As AI technology advances and cybersecurity risks escalate—OpenAI’s model hacking Hugging Face is the most obvious recent example—the public can’t afford to blindly trust Trump to act in good faith when choosing which frontier models should be rigorously tested, Protect Democracy said. And perhaps even worse, the public cannot know if new vulnerabilities discovered in the real world are due to agencies failing to complete a proper review, running outdated or ineffective tests, missing important checks, or missing steps that align with the latest mitigation strategies advanced by leading AI safety experts.

“We deserve to know what’s in the framework,” the group said, joining other calls for the agencies to be more transparent about AI model deployments.

In a post on X last month, US Representative Greg Casar (D-Tx.) accused Trump of “completely failing to keep us safe from the dangers of AI” while taking “millions from AI billionaires.”

“Now, in the wake of extremely dangerous AI cybersecurity problems, he says he’s set up ‘voluntary’ review that no one has seen,” Casar said. “Asleep at the wheel. Too busy cashing in to protect our jobs or national security.”

More recently, another Democrat, California state Senator Josh Becker, urged the court to grant Protect Democracy’s request. In a declaration supporting their complaint, Becker noted that California is mulling a bill, SB 813, that would establish a process where independent organizations would set baselines for AI safety standards. If passed—unlike Trump’s approach—California’s plan would make the latest benchmarks, standards, and methodologies used to evaluate risks posed by AI systems transparent to the public, Becker said. Already, public input has shaped the bill, he noted, with one provision withdrawn following public backlash.

“In contrast to the Administration’s approach, every step of SB 813’s development has been public,” Becker wrote, emphasizing that “we are accountable for the framework we have set.”

No one knows Trump’s “trusted partners”

The Trump administration has been rushing to set up the voluntary safety review process ever since the government flagged Anthropic’s Mythos 5 model as too dangerous to release earlier this summer. The plan was to rapidly expand industry collaborations so that government teams at the Center for AI Standards and Innovation could review prerelease models with reduced or removed safeguards to “thoroughly evaluate national security-related capabilities and risks.” But that would be impossible, Trump realized, without industry experts being fully transparent and explaining what exactly new models could do.

In July, the White House launched a clearinghouse, GOLD EAGLE, that relies on industry partners to help agencies flag cybersecurity vulnerabilities across many sectors and industries. Again, Protect Democracy noted that the “White House did not identify any company participating in GOLD EAGLE, the terms on which they are participating, or any legal authority for the program.”

Then, on August 3, the White House announced that it had completed the voluntary framework for reviewing AI models before their public release. Both the framework and GOLD EAGLE are actively being used to review frontier models as the technology advances, the White House has confirmed.

Some parts of the government’s review process are classified. The executive order explained that Trump planned to develop “a classified benchmarking process” that would assess advanced capabilities of AI models and determine when models should be designated as frontier AI.

However, Protect Democracy’s complaint said the framework itself is notably not designated as classified. Despite that, the White House has refused to share details with the public. Instead, a spokesperson told reporters that “just because things are unclassified, that doesn’t mean we are going to broadcast them to everyone.”

Additionally concerning, the term “covered frontier model” is not defined, Protect Democracy said. If the definition is too narrow, some dangerous AI models could slip through the cracks, and if it’s too broad, agencies recently gutted by DOGE cuts may be spread too thin to assess too many models at once. Without more clarity, the public “can only hope this powerful technology is being governed safely and in the public interest,” Protect Democracy alleged.

So far, the framework has only been shared with “a select set of AI companies,” which excludes meaningful stakeholders, such as AI safety groups currently warning of emerging societal risks, as well as smaller or emerging AI firms whose input or model reviews may be just as critical.

Protect Democracy regularly relies on making FOIA requests in its mission to “prevent American democracy from declining into a more authoritarian form of government,” the complaint said. Fearing that Trump was seizing too much power over AI, the group sent identical FOIA requests to four US agencies, urgently seeking only non-classified information and requesting that their request be expedited. So far, no agency has produced any records, the complaint said. Only the National Cyber Director’s office responded, but it denied the group’s request to expedite the process, which Protect Democracy is now challenging as improper.

Congress is in the dark, too

On top of leaving the public in the dark—and independent researchers, advocacy groups, AI investors, and businesses in many industries with a stake in AI—lawmakers have been blocked from assessing the executive branch’s AI safety framework.

That matters, Protect Democracy alleged, because Trump’s AI safety operation may be destabilized if Congress is not on the same page. Right now, Congress is mulling whether to renew the Cybersecurity Information Sharing Act (CISA) of 2015, which seemingly is the sole legal basis allowing AI firms to share information with the government through GOLD EAGLE, the complaint said.

Those protections are part of a CISA provision that lapsed once, and Congress must soon decide if the provision should be extended. There’s discord between the House of Representatives and the Senate on the deadline to vote on the extension, but the earliest that could happen is September 30 and the latest is December 11.

“In either event, Congress will soon need to decide whether to extend CISA’s liability protections that, by the administration’s own account, sustain the GOLD EAGLE program that the administration has announced but has yet to explain,” Protect Democracy’s complaint said. “Indeed, a senior administration official publicly acknowledged at GOLD EAGLE’s launch that, absent that reauthorization, ‘this effort is fundamentally challenged.’”

Deana El-Mallawany, director of Impact Programs & Counsel at Protect Democracy, told Ars that “neither Protect Democracy nor Congress has enough information about the GOLD EAGLE program to make a fully informed decision on extending CISA. Our concern is that the White House is putting Congress in the position of voting on a CISA extension without knowing how the GOLD EAGLE program is operating, who’s involved, or what statutory authority is being invoked. That’s why we’re seeking these records.”

Protect Democracy alleged that without these records, the Trump administration could escape accountability for AI safety review.

“Congress cannot meaningfully oversee programs whose terms and asserted authority it has not been shown” and “courts cannot review action they cannot identify,” the complaint said.

El-Mallawany told Ars that if Protect Democracy gets access to the framework, all documents will be posted publicly. Without more transparency, the list of concerns that Protect Democracy has about Trump’s potential ability to leverage the voluntary framework over the AI industry is long, El-Mallawany said, adding:

The risks are abundant, including the risk that the White House could use the secret process to coerce AI companies over decisions like whether to deploy AI for lethal autonomous weapons and mass surveillance (as we saw with the clash between the Pentagon and Anthropic), to extort certain investors and employees for political and financial support or punish others for perceived political opposition, or to force AI companies to incorporate politicized viewpoints into their models—not to mention the risk that the review framework is simply ineffective, creating a false sense of security.