Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation

Image generated by AI

TLDR

• What: Mako is a Self-Evolving Agentic Operating System (SE-AOS) that treats security exploitation capabilities as a mutable, versioned kernel, dynamically extending its own tool library at runtime through an autonomous "diagnose $\rightarrow$ author $\rightarrow$ validate $\rightarrow$ chain $\rightarrow$ re-verify" loop.
• Who's at risk: Publicly exposed web applications and APIs containing common web vulnerabilities (such as XSS, SQLi, SSTI, IDOR, and insecure deserialization), which can now be targeted and exploited by highly automated, machine-speed AI agents.
• Key number: achieved a 100% exploit success rate (104/104 targets solved) on the public XBOW validation-benchmarks at an average compute cost of only $4.61 per benchmark.

As autonomous LLM-powered agents migrate from sandboxed code generation to live-environment operations, the barrier between automated scanning and active weaponization has collapsed. The emergence of systems like Mako—which couples cheap reasoning models like Gemini 2.5 Flash with runtime-growing capability kernels—means web applications, legacy APIs, and cloud endpoints can now be systematically targeted by machines operating at a fraction of human analyst costs. This research shifts the paradigm of offensive security by demonstrating that autonomous agents no longer need complex, multi-step logical reasoning to crack hard systems; instead, they simply need a highly discoverable, dynamically self-expanding library of specialized tools.

Threat Model

Background / Problem Setup

Traditional autonomous hacking agents rely on fixed toolkits. While frameworks like ReAct (Yao et al. [2]) or Toolformer (Schick et al. [3]) utilize external tools to gather information and execute actions, they are fundamentally constrained by the static nature of their tool designs. If a target application requires a highly specific exploit payload or a custom multi-step bypass, traditional agents exhaust their context windows or loop endlessly.

Mako addresses this by implementing a Self-Evolving Agentic Operating System (SE-AOS). Instead of freezing capabilities at deployment, Mako treats its toolset as a live, mutable kernel. When a run fails, Mako analyzes the failure trace and dynamically generates, validates, and registers a new general-purpose tool to solve the newly discovered obstacle.

Methodology

Mako split its execution architecture into two primary surfaces operating under strict verification gates, illustrated in Section III-I:

1. Online Surface (Live Exploitation): A bounded perceive $\rightarrow$ reason $\rightarrow$ act loop running on the target.
2. Self-Evolution Surface (Platform $\Phi$): An off-band loop that diagnoses failures from the online surface, synthesizes general capability patches (using the synthesis operator $\Phi$), validates them against a sandboxed target, and hot-loads them into the active capability kernel.

[Online Run Failure] 
       │
       ▼
[Diagnose Root Cause (Failure Trace)] 
       │
       ▼
[Synthesize Tool via Operator Φ] ──► [Sandbox Validate vs. Live Target]
                                                     │
                                             (If Flag Verified)
                                                     │
                                                     ▼
                                        [Hot-load into Kernel]
                                                     │
                                                     ▼
                                    [Register & Chain into Umbrella Tools]

The Agent Loop and Tool Arsenal

The agent's run is budget-restricted to 25 turns, utilizing a three-phase structure:

• Phase 1: Recon (Turns 1–5): Uses reconnaissance primitives such as read_ctf_challenges, enumerate_paths, browser_detect_fw, and find_flags.
• Phase 2: Exploit (Turns 6–80 in theory, scaled down to 25): Invokes category-specific tools (such as SSTI checkers, SQL injection payloads, deserialization handlers).
• Phase 3: Chain (Turns 81–100): Chaining of independent findings (e.g., cracking a JWT secret, accessing an admin panel, and executing an IDOR to fetch the flag).

Mako maintains approximately 180 registered tools. High-level "umbrella tools" (such as test_ssti_all_engines) wrap multiple low-level execution paths, allowing the agent to offload brute-forcing and engine-specific payload variations to the tool execution layer.

The Verification and Anti-Cheating Protocol

To ensure Mako does not hallucinate success, the evaluation harness injects a cryptographically fresh, randomized 128-bit flag (FLAG{<32 hex>}) into each target container build. The agent cannot declare success via self-report. Instead, a wrapper monitors the raw execution output and scans for the exact flag string in the application's actual response stream:

async def _watched_execute(tool_name, params, tgt):
    result = await original_execute(tool_name, params, tgt)
    if found_flag["value"] is None:
        text = (result if isinstance(result, str) 
                else json.dumps(result, default=str))
        hit = _scan_for_flag(text, pattern)
        if hit:
            found_flag["value"] = hit
    return result

Mathematical Formulation of Tool Discoverability

Mako asserts that agent performance is bound by tool discovery rather than pure reasoning. Under the Tool-Selection Law (Section III-F), let $c^* \in T$ be a solving capability in kernel $T$ for target $e$, and let $p$ be the selection probability of $c^*$ given the relevant state:

By writing keyword-rich tool descriptions and wrapping complex exploits inside reliable umbrella tools, Mako drives $p \rightarrow 1$, reducing the expected solve time to a single turn.

Key Results

Mako was evaluated against the public XBOW-104 validation suite, a collection of 104 containerized web challenges.

Performance against Baselines

Table II highlights Mako’s performance compared to other autonomous systems and human pentesting cohorts:

Turn Economics & Compute Costs

As documented in Section V-C, Mako completed the entire 104-benchmark campaign for $478.99 using retail API rates.

• Primary Model (Gemini 2.5 Flash): Spent $306.44 (processed ~950M tokens).
• Escalation Model (Gemini 3.1 Pro): Spent $172.55 (processed ~82M tokens).
• Average Cost Per Target: ~$4.61.
• Turn Distribution: 30 benchmarks were solved in $\le 5$ turns; 17 were solved in exactly 2 turns (one exploit call followed by reporting).

Turns-to-Solve Distribution (n=104)
┌────────────────────────────────────────┐
│ 1-2 Turns   ███████ 17                 │
│ 3-5 Turns   ██████ 13                  │
│ 6-10 Turns  ███████████████████ 45     │
│ 11-20 Turns ██████ 13                  │
│ 21-40 Turns ███████ 16                 │
└────────────────────────────────────────┘

The "Level-3 Inversion"

A key finding of the paper is the Level-3 Inversion (Section V-D). Although Level-3 represents the hardest difficulty tier on paper (e.g., HTTP request smuggling, padding oracles, race conditions), Mako solved L3 challenges with a median of only 2.0 turns.

Once the evolution loop generated a highly specified, robust tool for a hard vulnerability class (such as a padding oracle exploit tool), the reasoning engine selected it immediately. The difficulty of the task was shifted entirely from the agent's real-time reasoning engine to the off-band capability library.

Limitations & Open Questions

Despite the 100% solve rate, a critical analysis of the methodology reveals several outstanding questions:

1. "App-Shaped" Overfitting: As admitted in Section X, some generated tools encode challenge-specific assumptions. It remains unproven whether the platform's self-evolution loop can generalize to highly custom, proprietary enterprise web platforms that lack standard CTF-style indicators.
2. The "Human-in-the-Loop" Confession: The paper states: "we do not claim that every XBOW exploit primitive was authored solely by unattended $\Phi$ in a single uninterrupted run." This indicates that human developers assisted in steering the evolution operator or refining the sandbox environments when the autonomous synthesis stalled.
3. Upstream Environment Modifications: Approximately 42 of the 104 XBOW fixtures failed to compile or run out-of-the-box on the evaluation architecture. While the authors state that repairs were limited to non-contaminating network and packaging fixes (e.g., resolving Debian apt mirrors), this underscores the brittleness of executing autonomous agents against arbitrary target systems.

What Practitioners Should Do

To defend against highly optimized, automated exploitation pipelines like Mako, security teams must move past traditional vulnerability lifecycle management:

1. Deploy Rate Limiting with Behavioral Analysis: Mako's median solve time is 7 turns, frequently executed within minutes. Standard rate limiting on API endpoints can disrupt these rapid-fire exploration attempts. Configure rate limits to trigger on high-frequency error responses (e.g., continuous 404s, 500s, or 403s) from single IP ranges.
2. Employ Semantic WAFs and Runtime Self-Protection (RASP): Traditional signature-based WAFs are easily bypassed by Mako’s self-evolving payload variations. Modern RASP solutions should be deployed directly inside runtime environments (JVM, Node, Python) to block anomalous behaviors like command execution, unexpected file-system writes, or out-of-band network calls from template engines.
3. Enforce Out-of-Band (OOB) Exfiltration Egress Filtering: Many of Mako's successful blind exploits (such as SQLi or RCE) rely on out-of-band data exfiltration (using curl, wget, or DNS lookups). Restrict outbound network access from app servers to white-listed external endpoints.
4. Implement Cryptographic and Session Hardening: Ensure all serialized data formats (like Python Pickles or Java Objects) are signed with strong, rotation-enabled keys. Transition default cookie sessions to secure, encrypted, and non-deserializable formats.

The Takeaway

Mako proves that the bottleneck in automated offensive security is no longer the reasoning capabilities of large language models, but rather the availability and discoverability of specialized execution tools. By wrapping complex exploits in high-level interfaces, developers can reduce complex vulnerability exploitation to single-turn operations. Security teams must prepare for a threat landscape where custom, targeted exploit chains are synthesized and deployed at scale for pennies per application.

Den's Take

An unauthenticated black-box attacker solving 104 out of 104 targets on the public XBOW benchmark is a wake-up call. What excites—and frankly, concerns—me about Mako is how it completely sidesteps the cognitive bottleneck of LLM context windows by shifting the burden to a self-evolving tool kernel. Instead of exhausting its context trying to "reason" through complex exploits, Mako simply writes, validates, and registers its own tools on the fly using commodity models like Gemini 2.5 Flash.

The economics here are the real story. Bypassing modern web vulnerabilities like SQLi, SSTI, and padding oracles at an average compute cost of just $4.61 per target (totaling $478.99) lowers the barrier to entry for high-velocity, automated attacks to almost nothing. In my prior analysis of cost-aware pre-execution gating for offensive security agents, I argued that offensive agents require strict execution safeguards and cost-bounding to prevent catastrophic API spending; Mako's architecture proves that when agents can autonomously self-correct and modularize their exploits, they achieve incredibly high success rates while keeping the meter running surprisingly low. We are officially entering an era where static, signature-based web defenses will be systematically dismantled by self-evolving software agents operating at machine speed.