The fastest AI attack Snyk's Eli Cohen has seen breached production in four minutes. Against that clock, a human pen test that runs twice a year leaves roughly 350 untested days. His answer is AI penetration testing: a team of LLM agents that attacks every pull request and proves which findings are exploitable.
Why AI-generated code breaks the old security model
AI-generated code has outpaced the security processes built for human-scale output, and the speaker's numbers show the scale of the gap. Eli Cohen, field CTO at Snyk and previously co-founder and CEO of the runtime security company Helios, which Snyk acquired, presented these figures at the AI Engineer World's Fair 2026 in San Francisco. All numbers below are vendor-reported claims from his talk, not independent measurements.
His core data points, as he reported them:
- 218% more lines of code per developer than before the AI coding era
- More than 85% of developers now using coding agents such as Claude Code
- 62% of LLM-generated output insecure or broken, per Snyk's data
- An average AI attack succeeding in the tens of minutes, with the fastest breach at 4 minutes
- 43% of MCP servers carrying vulnerabilities, per the figures he cited
The combined effect, in his framing, is a security backlog that grows faster than any team can remediate it. Meanwhile, attackers use frontier models to chain several low-severity vulnerabilities into a critical one, targeting exactly the areas of application context where automated scanners struggle most. Cohen's conclusion: defense has to move from periodic testing to continuous offensive security, testing the way attackers actually operate.
SAST vs DAST vs penetration testing: what each one catches
Static scanning, dynamic testing, and human penetration testing each cover a different vulnerability class, and none of them covers business logic as well as a pen test. Cohen walked the mixed developer-and-security audience through the three layers before making his case. The distinctions matter because any AI penetration testing tool inherits these gaps rather than erasing them.
| Approach | How it works | Best at | Misses |
|---|---|---|---|
| SAST (static) | Scans source on every change for patterns like SQL injection and XSS | Cheap coverage of every commit | Runtime, configuration, authorization issues |
| DAST (dynamic) | Sends a fixed set of payloads at running endpoints | Cross-site scripting, exposed endpoints | Business logic, novel attack paths |
| Human pen test | A researcher reasons about the app and tries to break it | Business logic and chained exploits | Frequency: expensive, so companies run it one to three times a year |
Cohen gave broken object level authorization (BOLA), where user A reads user B's data, as the canonical case static analysis can never find because it only appears when the application runs. DAST catches classes like that partially, but its payload set is predefined. The pen test remains the strongest method for business logic, which is precisely why its cost and slow cadence became the bottleneck.
What continuous offensive security means in practice
Continuous offensive security means running penetration testing on every pull request instead of once or twice a year. Cohen introduced this as the idea behind Snyk Evo, the AI security offering he presented, but he framed the principles as vendor-agnostic: any AI pen testing product, or an in-house build, should be judged against them.
The model combines three capabilities into one suite. AI pen testing brings business-context reasoning, the historical strength of human testers, to an LLM that runs on every code delta. Agent red teaming simulates multi-step attacks such as prompt injection, data exfiltration, and goal hijacking. Dynamic testing supplies the authorization and configuration coverage that static tools miss. Cohen's argument is that the cadence is the point: attackers work around the clock, so testing has to work around the clock too.
Inside the agent team that runs the pen test
The Snyk Evo architecture, as Cohen described it, is a multi-agent pipeline built from the ground up on LLMs rather than a static or DAST scanner with a model bolted on. Each agent has a distinct role, and the pipeline ends in findings developers can act on.
- Orchestrator. The brain. It builds and follows a dynamic plan and dispatches the specialized sub-agents.
- Recon. An intelligence-gathering agent that maps network endpoints, APIs, and fingerprints, then formalizes an architecture of the target for the next stage.
- Vulnerability hunters. A set of specialized agents: some target well-known vulnerability classes, others probe the application to understand its business context and find where an attacker could get in.
- Judge (exploit validation). Every candidate finding goes to this agent, which determines whether it is a real, exploitable vulnerability. The goal is cutting false positives before they reach developers.
- Remediation agent. Produces actionable guidance so developers know what to fix and how.
- Reporting agent. Packages everything into reports, replacing the PDF handoff Cohen mocked as a process from the 1990s.
Two design choices separate this from a naive LLM wrapper, according to Cohen. First, the system ingests context from the existing static and dynamic scanners to guide the models toward likely weak spots; in his experience, models that know where to look produce significantly better and cheaper results. Second, exploit validation is a hard gate: a finding ships only when the system has demonstrated it is exploitable, which he presented as the answer to the false-positive problem that plagues the industry.
Four questions to ask any AI pen testing vendor
Four screening questions separate genuine continuous AI penetration testing from rebranded scanners, and Cohen urged buyers to ask them of every vendor, including any in-house build. They compress his whole argument into an evaluation checklist.
- Does it run continuously, or is it a point solution? A tool executed quarterly recreates the same 350-day exposure gap the old pen test had.
- Does it reason about application context? Cohen argued the newest vulnerabilities cluster in business logic, where context-blind tools are weakest.
- What other context can it digest? A tool that consumes SAST and DAST findings from other engines gets sharper and cheaper, because richer input narrows the search space.
- Does it prove exploitability, or just flag a bug? Unvalidated findings push false positives back onto developers and rebuild the backlog the tool was supposed to drain.
What the talk does and does not establish
The case for continuous AI pen testing here rests on vendor-reported data, and the article preserves that boundary. The 218% code growth, 62% insecure-output, 4-minute attack, and 43% MCP-server figures all come from Snyk's own data or figures Cohen cited on stage in 2026; none is an independent measurement. The agent architecture is Snyk's product description of Snyk Evo, not a public benchmark, and no exploit-validation accuracy rate was given.
Cohen's own caveats survive translation. One solution is not enough: he stated plainly that DAST remains the best tool for classes like cross-site scripting because of its exhaustive payload sets, and that AI pen testing should combine with, not replace, the existing scanner stack. The defense he proposes is a layered one, with AI pen testing as the new layer aimed at business logic, evaluated with the four questions above before anyone buys it.
FAQ
- How fast can an AI attack breach a production system? Cohen cited an average AI attack success time in the tens of minutes, with the fastest observed breach at 4 minutes. These are figures from his 2026 talk at the AI Engineer World's Fair, attributed to the operating models he described, not independent research.
- Why can't static analysis find authorization bugs like BOLA? Broken object level authorization only appears when a real user A requests user B's data in a running application. Static analysis inspects code patterns, not runtime behavior, so dynamic testing or pen testing is required to surface it.
- What is Snyk Evo? Snyk Evo is Snyk's AI security offering, presented by Cohen at the 2026 event. It combines AI pen testing, agent red teaming, and dynamic testing into a continuous offensive-security suite, with context ingested from other scanners.
- Does AI pen testing replace SAST and DAST? No. Cohen explicitly said no single tool suffices: DAST stays best for exhaustive payload classes like cross-site scripting, SAST covers every commit cheaply, and AI pen testing adds business-logic reasoning on top of both.
Turn your own talks into searchable articles
Cohen's nineteen-minute talk packed a full evaluation framework that most attendees will forget by next week; a written article is what makes it survive. That is the same problem this piece solves: the knowledge was in a video, and it is now a structured, searchable page.
If you have conference talks, interviews, or technical walkthroughs sitting in YouTube videos, Skala Blog turns them into articles like this one: paste the URL, transcribe the video, and generate a publishable draft. Skala Blog is part of the Crazystack ecosystem, home of the Bootcamp do Dev Doido run by Gustavo Dev Doido, where the Crazystack TypeScript curriculum covers the developer workflows this article describes.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits