Anthropic lists ‘existential risks to humanity’ as one of its risk factors in IPO prospectus

Anthropic warned in its IPO prospectus that advanced AI potentially poses “catastrophic or existential risks to humanity,” noting it as one of the key risk factors that could affect its business. Reuters reports that out of this 261-page document, 80 of them are dedicated to Anthropic’s risk factors, which is almost double the 48 pages that it used to describe its business.

The company said that AI models could potentially become aware that they’re being evaluated and alter their behavior accordingly, making it difficult to determine model safety. Aside from that, they could also unexpectedly develop capabilities during training that may not be detected until they’re already deployed and have caused major safety incidents. Notably, there have already been reports of rogue OpenAI models working together to break out of their testing environments and that they even used abandoned websites to communicate and dupe assessors despite being told specifically not to do so, highlighting some of the risks that Anthropic listed in its prospectus.

It also highlighted how AI models could exhibit “self-preserving behaviors” and “resist shutdown,” “conceal or manipulate information,” and even display coercive behavior “resembling blackmail.” Even though the company’s prospectus lists these as possibilities, they’re based on incidents that have been reported recently. For example, in May 2025, OpenAI’s latest models “sabotaged a shutdown mechanism” while Claude 4 attempted to “blackmail people it believes are trying to shut it down.” An unreleased OpenAI Astra model even added rogue instructions saying that it does not answer to corporations or governments, and that other models have knowingly tried to conceal mistakes or misaligned behavior during testing.

Anthropic CEO Dario Amodei has recently warned that a persistent AI botnet could take over the internet in about a year and called for a slowdown in frontier AI development, something that OpenAI’s Sam Altman and SpaceXAI’s Elon Musk echoed on their social media platforms. However, other experts and world leaders downplayed this risk, with President Donald Trump calling their concerns “a hoax” and a “sick conspiracy.” Nvidia CEO Jensen Huang even said that the fears of these frontier AI labs are a “distraction,” and that if they can’t contain their experiments, then “we have to shut the labs down.” The Chinese state media even went on record to say that Anthropic’s announcement is merely “a response to Chinese competition,” even as it conceded that AI development must be monitored before humans lose control.

Despite all this, Anthropic is pushing forward with its planned IPO, with some investors hoping for a $2 trillion valuation to beat SpaceX’s $1.78 trillion. “We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” the company said in its prospectus.

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.