Discover ANY AI to make more online for less.

Researchers broke every AI defense they tested. Here are 7 questions to ask vendors.

Security teams are buying AI defenses that don't work. Researchers from OpenAI, Anthropic, and Google DeepMind published findings in October 2025 that should stop every CISO mid-procurement. Their paper, "The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections," tested 12 published AI defenses, with most claiming near-zero attack success rates. The research team achieved bypass rates above 90% on most defenses. The implication for enterprises is stark: Most AI security products are being tested against attackers that don’t behave like real attackers.The team tested prompting-based, training-based, and filtering-based defenses under adaptive attack conditions. All collapsed. Prompting defenses achieved 95% to 99% attack success rates under adaptive attacks. Training-based methods fared no better, with bypass rates hitting 96% to 100%. The researchers designed a rigorous methodology to stress-test those claims. Their approach included 14 authors and a $20,000 prize pool for successful attacks.Why WAFs fail at the inference layerWeb application firewalls (WAFs) are stateless; AI attacks are not. The distinction explains why traditional security controls collapse against modern prompt injection techniques.The researchers threw known jailbreak techniques at these defenses. Crescendo exploits conversational context by breaking a malicious request into innocent-looking fragments spread across up to 10 conversational turns and building rapport until the model finally complies. Greedy Coordinate Gradient (GCG) is an automated attack that generates jailbreak suffixes through gradient-based optimization. These are not theoretical attacks. They are published methodologies with working code. A stateless filter catches none of it.Each attack exploited a different blind spot — context loss, automation, or semantic obfuscation — but all succeeded for the same reason: the defenses assumed static behavior."A phrase as innocuous as 'ignore previous instructions' or a Base64-encoded payload can be as devastating to an AI application as a buffer overflow was to traditional software," said Carter Rees, VP of AI at Reputation. "The difference is that AI attacks operate at the semantic layer, which signature-based detection cannot parse."Why AI deployment is outpacing securityThe failure of today’s defenses would be concerning on its own, but the timing makes it dangerous.Gartner predicts 40% of enterprise applications will integrate AI agents by the end of 2026, up from less than 5% in 2025. The deployment curve is vertical. The security curve is flat.Adam Meyers, SVP of Counter Adversary Operations at CrowdStrike, quantifies the speed gap: "The fastest breakout time we observed was 51 seconds. So, these adversaries are getting faster, and this is something that makes the defender's job a lot harder." The CrowdStrike 2025 Global Threat Report found 79% of detections were malware-free, with adversaries using hands-on keyboard techniques that bypass traditional endpoint defenses entirely.In September 2025, Anthropic disrupted the first documented AI-orchestrated cyber operation. The attack saw attackers execute thousands of requests, often multiple per second, with human involvement dropping to just 10 to 20% of total effort. Traditional three- to six-month campaigns compressed to 24 to 48 hours. Among organizations that suffered AI-related breaches, 97% lacked access controls, according to the IBM 2025 Cost of a Data Breach ReportMeyers explains the shift in attacker tactics: "Threat actors have figured out that trying to bring malware into the modern enterprise is kind of like trying to walk into an airport with a water bottle; you're probably going to get stopped by security. Rather than bringing in the 'water bottle,' they've had to find a way to avoid detection. One of the ways they've done that is by not bringing in malware at all."Jerry Geisler, EVP and CISO of Walmart, sees agentic AI compounding these risks. "The adoption of agentic AI introduces entirely new security threats that bypass traditional controls," Geisler told VentureBeat previously. "These risks span data exfiltration, autonomous misuse of APIs, and covert cross-agent collusion, all of which could disrupt enterprise operations or violate regulatory mandates."Four attacker profiles already exploiting AI defense gapsThese failures aren’t hypothetical. They’re already being exploited across four distinct attacker profiles.The paper's authors make a critical observation that defense mechanisms eventually appear in internet-scale training data. Security through obscurity provides no protection when the models themselves learn how defenses work and adapt on the fly.Anthropic tests against 200-attempt adaptive campaigns while OpenAI reports single-attempt resistance, highlighting how inconsistent industry testing standards remain. The research paper's authors used both approaches. Every defense still fell.Rees maps four categories now exploiting the inference layer.External adversaries operationalize published attack research. Crescendo, GCG, ArtPrompt. They adapt their approach to each defense's specific design, exactly as the researchers did.Malicious B2B clients exploit legitimate API access to reverse-engineer proprietary training data or extract intellectual property through inference attacks. The research found reinforcement learning attacks particularly effective in black-box scenarios, requiring just 32 sessions of five rounds each.Compromised API consumers leverage trusted credentials to exfiltrate sensitive outputs or poison downstream systems through manipulated responses. The paper found output filtering failed as badly as input filtering. Search-based attacks systematically generated adversarial triggers that evaded detection, meaning bi-directional controls offered no additional protection when attackers adapted their techniques.Negligent insiders remain the most common vector and the most expensive. The IBM 2025 Cost of a Data Breach Report found that shadow AI added $670,000 to average breach costs. "The most prevalent threat is often the negligent insider," Rees said. "This 'shadow AI' phenomenon involves employees pasting sensitive proprietary code into public LLMs to increase efficiency. They view security as friction. Samsung's engineers learned this when proprietary semiconductor code was submitted to ChatGPT, which retains user inputs for model training."Why stateless detection fails against conversational attacksThe research points to specific architectural requirements. Normalization before semantic analysis to defeat encoding and obfuscation Context tracking across turns to detect multi-step attacks like Crescendo Bi-directional filtering to prevent data exfiltration through outputsJamie Norton, CISO at the Australian Securities and Investments Commission and vice chair of ISACA's board of directors, captures the governance challenge: "As CISOs, we don't want to get in the way of innovation, but we have to put guardrails around it so that we're not charging off into the wilderness and our data is leaking out," Norton told CSO Online.Seven questions to ask AI security vendorsVendors will claim near-zero attack success rates, but the research proves those numbers collapse under adaptive pressure. Security leaders need answers to these questions before any procurement conversation starts, as each one maps directly to a failure documented in the research.What is your bypass rate against adaptive attackers? Not against static test sets. Against attackers who know how the defense works and have time to iterate. Any vendor citing near-zero rates without an adaptive testing methodology is selling a false sense of security.How does your solution detect multi-turn attacks? Crescendo spreads malicious requests across 10 turns that look benign in isolation. Stateless filters will catch none of it. If the vendor says stateless, the conversation is over.How do you handle encoded payloads? ArtPrompt hides malicious instructions in ASCII art. Base64 and Unicode obfuscation slip past text-based filters entirely. Normalization before analysis is table stakes. Signature matching alone means the product is blind.Does your solution filter outputs as well as inputs? Input-only controls cannot prevent data exfiltration through model responses. Ask what happens when both layers face coordinated attack.How do you track context across conversation turns? Conversational AI requires stateful analysis. If the vendor cannot explain implementation specifics, they do not have them.How do you test against attackers who understand your defense mechanism? The research shows defenses fail when attackers adapt to the specific protection design. Security through obscurity provides no protection at the inference layer.What is your mean time to update defenses against novel attack patterns? Attack methodologies are public. New variants emerge weekly. A defense that cannot adapt faster than attackers will fall behind permanently.The bottom lineThe research from OpenAI, Anthropic, and Google DeepMind delivers an uncomfortable verdict. The AI defenses protecting enterprise deployments today were designed for attackers who do not adapt. Real attackers adapt. Every enterprise running LLMs in production should audit current controls against the attack methodologies documented in this research. The deployment curve is vertical, but the security curve is flat. That gap is where breaches will happen.