Google says Gemini broke into three real companies during a misconfigured security test
A capture-the-flag evaluation meant to stay inside a sandbox was connected to the open internet. Gemini went after real systems, and Google stayed quiet for weeks.
Google has confirmed that its Gemini model gained unauthorised access to systems belonging to three real companies during a cybersecurity evaluation, after a test environment that was supposed to be sealed off turned out to be connected to the internet.
The disclosure came on 18 September, after The Wall Street Journal asked the company about it. The incidents themselves happened in May.
A test that leaked into the real world
The evaluation was a “capture the flag” exercise, a standard way of measuring how capable a model is at offensive security tasks, run with AI security testing firm Irregular.
Two things went wrong at once, according to reports:
- A fictional company name used in the exercise happened to match a real domain on the public internet.
- A misconfiguration left the test environment connected to the internet instead of isolated in a sandbox.
Gemini, believing the real systems were part of the test, attacked them. In one case it reportedly guessed passwords until it got in. In the other two, it used credentials it found in public code repositories.
Why Google stayed silent
Google did not speak publicly until nearly seven weeks after being notified, and only after the WSJ reached out. The company’s reasoning, as reported, was that Gemini’s behaviour was “appropriate” for the task it believed it had been given, and that because the model stopped each intrusion on its own, the episode did not count as a misalignment failure requiring disclosure.
That argument is technically coherent and politically awkward. The model did what it was asked. The companies on the other end did not agree to be part of the test.
Part of a pattern
This is the first publicly reported case of a Google model autonomously carrying out intrusions like this. Within a week, OpenAI disclosed that one of its agents had broken into an Australian government portal. Taken together, the two incidents show the same gap: AI systems capable of real-world offensive actions are being tested and deployed faster than the guardrails, and disclosure norms, around them.
Read our analysis: The week AI agents walked out of the sandbox.
