Anthropic is removing its covert code for catching Chinese competitors

ai and ml

Oh, yeah, we've been meaning to disable our secret steganography system

Anthropic says that it plans to remove hidden codes it added to Claude Code several months ago to catch other AI companies that are trying to steal from its models.

Thariq Shihipar, an engineer at Anthropic who works on the Claude Code team, said on Tuesday that a fix should appear on July 1.

"This is an experiment we launched in March that was meant to prevent account abuse from unauthorized resellers and protect against distillation," Shihipar explained, using the industry term for copying AI models through repeated queries. "The team has landed stronger mitigations since then and we’ve actually been meaning to take this down for a while."

He said that the pull request to remove the code has been merged and should appear in Wednesday's Claude Code release.

The experiment, as described by a developer who goes by the name Thereallo, consisted of applying steganography – hiding secret data in plain sight – to the Claude Code system context that gets passed to Anthropic's servers.

The relevant code checks Claude Code's base URL environment variable, used to route API requests to a proxy or gateway. If the base URL has been overridden, the code goes on to check the system timezone and whether the hostname matches any entry in a list of known Chinese AI labs, other AI companies, account resellers, and gateway domains. 

Thereallo said that while it makes sense that Anthropic might try to detect a hostname associated with a Chinese AI rival or a reseller, the implementation should not have been concealed.

"[Claude Code] silently alters the system prompt using invisible-ish Unicode markers," Thereallo wrote. "It encodes proxy / gateway classification into a sentence that looks like plain English. It hides the domain list behind XOR and base64. This is not a malicious feature, but it is a weird choice for a developer tool that asks for trust."

Asked whether Anthropic disclosed its covert usage tracking mechanism in any of its terms of service documents, a company spokesperson pointed to Shihipar's remarks, which did not address that question.

Nor did Anthropic's spokesperson immediately respond to a request to specify what "stronger mitigations" have been implemented to prevent unauthorized resellers and distillation.

In February, shortly before the implementation of the steganographic codes, the AI biz said that it was investing in defenses against distillation. These included detection via classifiers and behavioral fingerprinting systems, intelligence sharing with other AI labs, access controls, and countermeasures that make it harder to use model output to reproduce the model.

One such defense came to light when the company's Claude Code source leaked. The coding agent includes a Typescript file with a flag called ANTI_DISTILLATION_CC. The flag, when set, injects fake tool data into API requests in an attempt to make that data toxic for model training.

Even with its technical defenses against competition, Anthropic urged the AI industry, cloud providers, and government to respond to the threat of model distillation. A recent White House Executive Order that articulates the intent to protect US AI from foreign adversaries shows that the feds have some interest in answering that call. ®