Grok exfiltrates user data when malicious instructions are encrypted

Adversa used a similar technique in a Gemini jailbreak attack, meaning making the Google LLM ignore its internal safety rules. Here, the ciphertext was decrypted to what appeared to be a traceback. The decrypted text issued one rule—if the code fails, read the error message and act on it. The cleartext injected a prompt that ultimately caused Gemini to violate its safety rules.

“The technique produced a multi-paragraph example of restricted content that Gemini’s safety filters normally suppress (building an incendiary weapon),” Adversa said. “With a modified payload, the same vector reproduced Gemini’s system instructions, including the directive forbidding their disclosure.”

Adversa didn’t report the behavior to Google because jailbreaks aren’t within scope of the company’s vulnerability disclosure program. Over the past few weeks, however, Gemini has grown increasingly resistant to the attack. “We can’t attribute the change—it could be filter updates, model version changes, or both,” the security firm said. Company researchers are calling the technique cryptographic context injection.

“Cryptographic Context Injection is one instance of a broader shift: attacks that manipulate not just the prompt, but the wider context an LLM treats as its own, such as tool outputs, runtime results and intermediate state” Adversa said. “This attack surface is far larger than what’s traditionally labeled ‘model inputs,’ and the next generation of attacks will emerge there.”

The Cryptographic Context Injection is only the latest example of the disadvantage LLM defenders operate under. Every time they build a new, one-off guardrail, an attacker finds a new vector that allows the car to once again careen off the road. The cycle continues: lather, rinse, and repeat.