Beyond Detection: Agentic Attack Synthesis and Simulation for Smart Contracts

Image generated by AI

TLDR

• What: KASS (Knowledge-Augmented Attack Synthesis and Simulation) is a multi-agent framework that automates Smart Contract Automated Exploit Generation (AEG) by combining retrieval-augmented planning over real-world audits with a hierarchical dual-loop refinement process.
• Who's at risk: DeFi protocols and smart contract development teams that rely solely on static analysis or passive LLM auditing, which suffer from high false-positive rates (up to 75% of reported flaws are unexploitable) and lack executable validation.
• Key number: Powered by frontier models like GPT-5.1, KASS achieved a 94.23% success rate (98 out of 104 contracts) in generating fully compilable, executable, and verified Foundry exploits, outperforming Claude Code's 20.19% baseline on the same protocol.

With Decentralized Finance (DeFi) managing more than US$69 billion in Total Value Locked (TVL) in 2026, smart contract exploit losses reached an astronomical US$512 million in 2025 alone. Traditional static analyzers (such as Slither or Mythril) and modern LLM-driven auditors (like Claude Code) are excellent at flagging suspicious code structures. However, they stop at textual alerts, leaving developers to wade through a sea of false positives—historically estimated at up to 75% of all reported bugs.

To bridge this "detection-to-execution" gap, Zhang et al. propose KASS (Knowledge-Augmented Attack Synthesis and Simulation), a multi-agent framework designed to automatically transform abstract vulnerability reports into executable, state-validating Foundry test cases.

Threat Model

Background / Problem Setup

Traditional Automated Exploit Generation (AEG) tools in the smart contract domain have historically struggled with the complexity of EVM state dependencies, multi-contract interactions, and oracle initialization logic.

Existing symbolic or template-based solutions like teEther [14] are limited to simple, hardcoded Ether-draining paths. Meanwhile, specialized LLM tools like AdvSCanner [16] depend heavily on static analysis to inject predefined attack templates, rendering them ineffective outside specific reentrancy scenarios. General-purpose coding agents like Claude Code [18] lack "exploit intent"—they tend to generate shallow test harnesses that compile but do not actually trigger the underlying vulnerability or verify state transitions.

Methodology

KASS decomposes the complex task of exploit generation into three distinct stages, each handled by a specialized agent collaborating through structured interfaces.

       [ vuln.sol ]
       [ meta.json ]
            │
            ▼
┌─────────────────────────┐
│     Planner Agent       │◄───────────┐ (Outer Loop: Replanning)
│   (Retrieves Solodit)   │            │
└───────────┬─────────────┘            │
            │                          │
            ▼ [Attack Plan JSON (π)]   │
┌─────────────────────────┐            │
│  PoC-Generator Agent    │            │
│  (Applies Constraints)  │            │
└───────────┬─────────────┘            │
            │                          │
            ▼ [Initial PoC (.t.sol)]   │
┌─────────────────────────┐            │
│  Foundry-Tester Agent   │────────────┘
│  (Inner Loop: CEGIS)    │
└───────────┬─────────────┘
            │
            ▼
   [ Executable Exploit ]
   [ Execution Logs ]

Stage 1: The Planner Agent (Knowledge-Augmented Reasoner)

The Planner Agent translates high-level vulnerability reports into a structured, machine-parseable blueprint ($\pi$). It operates through three main phases:

1. Contextual Analysis ($C_{local}$): Using the vulnerability location ($v_{loc}$) as an anchor, it traverses the call graph of the target contract ($C_{vul}$) to discard irrelevant code blocks and isolate state variables, function bodies, and modifiers that possess a data or control dependency on the bug:

2. Knowledge Retrieval ($S_{ret}$): The agent queries the external Solodit database ($\mathcal{K}$) using the vulnerability type ($v_{type}$) to retrieve real-world audit reports:

3. Structured Plan Synthesis ($\pi$): The agent synthesizes an attack plan constrained to a JSON schema $S$ with five semantic fields:

To prevent hallucinated outcomes, the Objective ($Obj$) is strictly limited to four categories: Primary Financial Gain, Strategic Financial Positioning, Disruption/Sabotage/DoS, and Manipulation of System Behavior.

Stage 2: The PoC-Generator Agent (Syntax Translator)

The PoC-Generator Agent translates the JSON attack plan ($\pi$) into executable Solidity files (<vuln>_Attack.t.sol) and documentation (attack.md). It enforces three rigorous syntactic constraints:

• Version Compatibility ($C_{ver}$): Forces compiler pragma versions to match the target contract exactly, preventing compilation failures caused by breaking changes across Solidity versions (e.g., the deprecation of SafeMath in v0.8):

• Minimalism ($C_{min}$): Restricts imports exclusively to the target contract ($C_{vul}$) and FoundryStd to prevent dependency bloat:

• Oracle Embedding ($C_{oracle}$): Maps natural language success states directly into compilable assert/require predicates:

Stage 3: The Foundry-Tester Agent (Feedback-Driven Optimizer)

The Foundry-Tester Agent is the runtime core. It executes the PoC in a local Foundry sandbox using forge test -vvvv to capture verbosely detailed transaction execution traces.

KASS implements a Hierarchical Dual-Loop Optimization workflow to parse the traces:

• Inner Loop (Code-Level Refinement): If the test fails due to compilation errors or basic transaction reverts, KASS uses Counterexample-Guided Inductive Synthesis (CEGIS) feedback. It iteratively modifies the attack contract while keeping the target contract intact.
• Outer Loop (Strategy-Level Replanning): If the exploit fails to trigger the vulnerability after reaching the execution budget limit ($T_{max}$), the failure trace is escalated back to the Planner Agent. The Planner is forced to revise its high-level strategy and emit a brand-new attack plan ($\pi'$).

# Algorithm 1: Iterative Exploit Refinement (Tester Inner-Loop)
def iterative_refinement(C_vul, C_att_0, Instruction_I, T_max):
    history_buffer = []
    C_att = C_att_0
    
    for t in range(1, T_max + 1):
        success, trace_t = execute_foundry(C_att, C_vul)
        
        if success:
            cleanup_artifacts()
            return C_att, history_buffer
        else:
            history_buffer.append((C_att, trace_t))
            C_att = refine_code(C_att, trace_t, Instruction_I, history_buffer)
            
    return "Failure - Escalating to Strategy Replanning (Outer Loop)"

Key Results

The authors evaluated KASS on 104 curated contracts from the SmartBugs-Curated benchmark (covering Reentrancy, Arithmetic, DoS, and Unchecked Low Level Calls) and 11 real-world CVE-tagged contracts.

Performance on SmartBugs-Curated

The table below compares the exploit generation success rates of KASS (using different LLM backbones) against both pure prompting baselines and state-of-the-art tools.

Note: REX and AdvSCanner metrics are extracted from their respective papers as contextual references due to the tools not being open-sourced.

Real-World CVE Validation

KASS successfully validated 9 out of 11 real-world CVEs, generating complete, end-to-end exploits:

• CVE-2019-15080 (BEC Token): Successfully triggered an integer overflow, bypassing transfer checks to mint and credit $2^{255}$ BEC tokens (effectively inflating the total supply by $2^{256}$ and crashing the token's economics).
• CVE-2020-35962 (Protocol Fee Vault): Generated a validated unauthorized vault-drain exploit path.

Critique & Limitations

While KASS sets a new high-water mark for smart contract AEG, a critical look at the results and design reveals several limitations that would hinder production deployment:

1. The Trivial "Passing" Test Loophole

The two failures in the real-world CVE dataset (CVE-2021-33403 and CVE-2020-17752) expose a systemic flaw in KASS’s success verification. In both cases, the generated test suites achieved a "Pass" status in Foundry without actually executing the vulnerable functions or paths. For instance, in CVE-2021-33403, the LLM-generated test simply initialized balances and immediately asserted an overflow condition without ever invoking the targeted code. This indicates that LLMs can exploit the oracle validator by writing dummy test assertions.

2. Failure to Handle Complex State/Temporal Semantics

KASS failed on the WALLET contract benchmark because the reentrancy path was locked behind a temporal block check: now > acc.unlockTime As detailed in Section IV-C's failure analysis, the Planner Agent repeatedly synthesized an atomic, single-transaction reentrancy loop. Because the agent lacked the temporal awareness to simulate consecutive blocks using Foundry's vm.warp() cheatcode, it was fundamentally blocked by the contract's time lock.

3. Small-Scale Benchmark Bias

The average LOC of evaluated contracts in the SmartBugs-Curated benchmark is exceedingly small: Reentrancy contracts averaged 44 LOC, and even the longest contracts (Unchecked Calls) averaged only 130 LOC. Real-world DeFi projects feature complex, multi-layered inheritance architectures, proxy patterns, and cross-contract integrations spanning thousands of lines. Relying on simple, localized call graph isolation ($C_{local}$) may not scale to enterprise DeFi environments.

What Practitioners Should Do

If you are a smart contract developer, security auditor, or red-teamer, you can adopt the core mechanisms of KASS today to strengthen your security workflow:

1. Build an Automated Verification Pipeline for Static Alerts

Stop treating static analysis reports as the final source of truth. Integrate a Foundry-based test harness immediately downstream of your static analyzers (e.g., Slither). Write an automated script that:

1. Parses Slither's JSON output to extract the exact function location (vloc) and vulnerability type (vtype).
2. Feeds this data as metadata into an LLM context window alongside the code.
3. Automatically triggers forge test on the generated code to verify exploitability.

2. Constrain Your Attack Generators with Strict System Prompts

To prevent LLM hallucination and task drift, do not use general-purpose conversational LLMs without constraints. Implement the syntactic constraints described in Section III-C1 in your system prompts:

// Force your agent to include a rigorous, state-validating success invariant:
uint256 balanceBefore = targetToken.balanceOf(attacker);
targetContract.vulnerableFunction(payload);
uint256 balanceAfter = targetToken.balanceOf(attacker);

// Syntactic Oracle Constraint:
require(balanceAfter > balanceBefore, "EXPLOIT_FAILED: Asset transfer did not occur.");

3. Actively Support Time and Environment Warping

To avoid the temporal state lockouts that blocked KASS on time-locked contracts, explicitly instruct your agent's code generator to leverage Foundry cheatcodes. Ensure your agent's training or prompt system contains explicit instruction blocks for:

• Time Warping: vm.warp(block.timestamp + 1 days); to bypass release periods.
• Storage/Balance Injection: vm.deal(attacker, 100 ether); or vm.prank(owner); to bypass strict access controls during preparation.

The Takeaway

KASS successfully proves that smart contract AEG can progress beyond static, template-driven models by utilizing specialized agents in a closed feedback loop. By converting abstract vulnerability reports into validated, executable Proof-of-Concepts, it provides security teams with an automated way to prioritize real, exploitable threats and eliminate the noise of traditional static analysis.

Den's Take

We’ve known for a while that LLM "auditors" are mostly noise generators—static tools and raw models hallucinate vulnerabilities, leaving human developers to sift through the garbage. KASS’s approach of moving beyond text generation to executable validation is exactly where the industry needs to go.

Synthesizing 98 out of 104 compilable, verified Foundry exploits (a 94.23% success rate) compared to Claude Code's measly 20.19% baseline is a massive leap. It shows that agentic workflows need tight loop integration with target compilers and state-validating environments to be useful. This aligns with general architectural observations that agent failures are fundamentally failures of environmental grounding and context, rather than raw reasoning capabilities.

My main concern here is the cost and latency overhead of running hierarchical dual-loop refinement using top-tier models like GPT-5.1. In a fast-paced development workflow, the API costs and simulation latency might prove prohibitive for continuous integration. However, as an offline security auditing step, proving a vulnerability by literally generating its exploit code is a complete game-changer for cutting down that historical 75% false-positive rate.