ASAPAGI Soon As Possible · Deep reads on AI & tech
Article

OpenAI GPT-Red: An Automated Red Team Trained to Hack Its Own Models

2026-07-16 · 6 min read

OpenAI unveiled GPT-Red on July 15, 2026, an automated red-teaming model that hunts for vulnerabilities in the company's own systems. OpenAI trained GPT-Red through a self-play loop in which attacker and defender models competed against each other, and the company stated that more than 90 percent of its strongest attacks succeeded against GPT-5 in August 2025 while fewer than 23 percent worked against GPT-5.6 in July 2026. In a matchup against human red-teamers, GPT-Red cracked 84 percent of scenarios versus the humans' 13 percent.

The Self-Play Training That Pitted Attack Against Defense

GPT-Red was built to automatically find prompt injections and other hijacking attacks. OpenAI trained it in a self-play loop that put attacker and defender models in one environment and made them compete, where the attacker's goal was to break the other model and the defender's goal was to protect itself. Training happened in a simulated dojo that mimicked real large language model deployments, complete with the tools real agents use such as web browsing, email and calendar access, and code editing. OpenAI said it poured an unprecedented amount of compute into this safety work. The researchers credited as co-creators include Nikhil Kandpal, Dylan Hunn, and Chris Choquette-Choo.

What separates this from conventional red teaming is that the attacker is a trainable model rather than a person. Traditional red teams have security experts hand-craft attack scenarios and test them one by one. Humans are creative but slow, and there are only so many of them to sweep the same class of vulnerability repeatedly. Pitting an attack model against a defender, by contrast, pushes the number of attempts far past what human hands allow, and the attack evolves as the defense hardens. It is the self-play dynamic that made AlphaGo strong, transplanted into the safety domain. Moving vulnerability search from human labor to compute is the central axis of this release.

Reading the Drop From 90 Percent to 23 Percent

The most striking figure is the gap between GPT-5's 90-plus percent and GPT-5.6's under-23 percent against the same attack tool. On the surface this reads as a signal that the newer model is far more resistant to prompt injection. If the share of successful attacks fell to below a quarter in a single year, the defense side has arguably hardened in real terms. Still, reading that number straight as "now it is safe" calls for caution. Both 90 percent and 23 percent are internal measurements OpenAI obtained by running its own tool against its own models. Because the attack and defense models come from the same team, this score does not cover attack classes GPT-Red has not yet imagined.

More important is that 23 percent is not zero. It means close to one in four strong attacks still works, and when an agent holds real permissions such as email, payments, or code execution, a single success can cause harm. A higher defense rate does not mean risk has vanished so much as that the cost of attacking has risen. The improvement is welcome, but it should not be stretched into a safety guarantee for deployment.

The "Fake Chain of Thought" That Exposed an Agent's Weak Spot

Among the attacks GPT-Red found was a new class researchers had not previously known: the "fake chain of thought" attack, which plants false information in a model's working memory. Chris Choquette-Choo described it this way: "It's like if I told you that 1+1=3 and that you have verified this already." It pushes fake intermediate steps that look self-derived into the model, tricking it into believing verification is already done.

This attack matters because it collides head-on with a recent trend in agent design. Many of today's agents raise reliability by unfolding long chains of thought, or intermediate reasoning. If that intermediate process itself becomes a target for manipulation, then a model "showing its work" does not automatically mean safety. It is a paradox: the more reasoning is exposed, the larger that exposed surface can become as an attack surface. For the practice of treating prompt-injection defense purely as an input-filtering problem, this finding is a clear warning.

Beating Human Red Teams, and the Limits That Remain

GPT-Red actually broke into Vendy, an AI vending-machine agent in OpenAI's office, manipulating prices and canceling orders. Compared with 2025 human red-team experiments, GPT-Red cracked 84 percent of scenarios while humans managed 13 percent, showing that an automated attacker outpaces people in speed and scale. But OpenAI disclosed the limits too. GPT-Red was weak at multi-turn conversational attacks and had limited capability with image-based prompt injections that hide instructions in pictures. OpenAI decided not to release the model publicly.

AI security analyst Jessica Ji noted that despite this automation, human expertise will still be very important. An automated red team can sweep more scenarios faster than people, yet defining new classes of threat and interpreting results remains a human job. The 84 percent score shows automation's strength, while the weaknesses in multi-turn and image attacks show that this strength is bound to specific conditions.

What It Means for Deploying AI Agents in Korea

The message for Korean practitioners is clear. When a domestic company moves beyond a chatbot to an agent that touches email, payments, and internal systems, the vending machine GPT-Red broke is not someone else's problem. The moment an agent holds real permissions, prompt injection stops being a demo blemish and becomes an operational incident. The fake chain of thought attack in particular shows that a design trusting the model's reasoning does not by itself guarantee safety, so keeping human approval at each action step and enforcing least-privilege before granting agents authority is the safer path.

At the same time, this release carries a two-sided truth: as defense tools become automated, so do attack tools. OpenAI chose not to release GPT-Red, but the possibility of a similar self-play method being used for non-defensive ends has not disappeared. Any organization wiring an AI agent into a service would do well to build a process that validates scenarios, including multi-turn and image injections, in its own environment rather than accepting a vendor's defense-rate number at face value.


Sources: MIT Technology Review · The Next Web · OpenAI (2026-07-15)

ASAP — AGI Soon As Possible

AI & tech,
read in depth

Beyond the headlines — into the context and the structure

AGI Soon As Possible · asapai.co.kr

← All posts