AutoTrace: From Patches to Triggers via Agentic Interprocedural Exploration

Image generated by AI

TLDR

• What: AutoTrace is an agentic framework that localizes deep interprocedural vulnerability triggers by combining LLM-guided call-graph exploration with rigid, deterministic Code Property Graph (CPG) verification gates.
• Who's at risk: Enterprise security pipelines, automated patch-validation tools, and IDE-integrated coding assistants (e.g., Copilot Enterprise, Cursor) that rely on ungrounded LLMs or shallow, intra-function heuristic scanners.
• Key number: AutoTrace achieves 75.0% VulnHit on the full InterPVD benchmark, outperforming the previous state-of-the-art VulTrigger (69.8%) and nearly doubling a standard LLM+RAG baseline (37.8%).

When security patches are deployed to fix critical vulnerabilities, the code that introduces the fix is rarely where the exploit actually occurs. In complex C/C++ codebases—such as the Linux kernel, QEMU, or FreeType—the actual crash site or unsafe operation (the "trigger") often lies three or four call layers deep in a completely different file. Traditional static analyzers and deep learning line-rankers fail to track this because they lack either the interprocedural reasoning or the semantic comprehension of custom code wrappers.

AutoTrace addresses this challenge by pairing frontier LLMs (specifically Gemini-3-Flash) with deterministic, machine-verifiable graph-traversal constraints. It navigates a dynamically expanded Code Property Graph (CPG) to locate vulnerabilities with mathematical backing, ensuring that no vulnerability claim relies solely on ungrounded model judgment.

Threat Model

Unlike conventional attack papers, AutoTrace defines its threat model around the integrity of the automated scanning pipeline itself, protecting the analysis tool from malicious source-code manipulation.

Background & Problem Setup

The core difficulty in vulnerability localization is distinguishing the patched statement (where a guard is added) from the trigger statement (where the unsafe operation executes).

[ cf2_interpT2CharString() ] <-- Patch: bounds guard added here
          |
          v (calls)
[ cf2_doStems() ]
          |
          v (calls)
[ cf2_stack_getReal() ]     <-- Trigger: stack->buffer[idx] (Unsafe Read)

As demonstrated in the paper's motivating example (CVE-2014-9659, Figure 1), a patch in cf2_interpT2CharString introduces a bounds check, but the actual trigger is three hops away inside cf2_stack_getReal.

The table below contrasts how AutoTrace compares to previous approaches:

Methodology

AutoTrace functions as a multi-stage pipeline, splitting semantic reasoning (LLM) from deterministic execution checks (Joern CPG engine).

   [ Commit Diff ] 
          │
          ▼
┌─────────────────────────────────┐
│ Stage 1: Critical Variable Ext. │ ──> Extracts target variables (C_Δ)
└─────────────────────────────────┘
          │
          ▼
┌─────────────────────────────────┐
│ Stage 2: Patch-Scoped CPG       │ ──> Dynamically builds local AST/PDG/CFG
└─────────────────────────────────┘
          │
          ▼
┌─────────────────────────────────┐
│ Stage 3: Agentic Exploration    │ ──> Best-first frontier queue traversal
└─────────────────────────────────┘
          │
          ▼
┌─────────────────────────────────┐
│ Stage 4: Multi-Layer Verifier   │ ──> Admissibility & Corroboration Gates
└─────────────────────────────────┘
          │
          ▼
   [ Verified Trigger ]

Stage 1: Critical-Variable Extraction

Using Agent 1 (configured with the prompt shown in Appendix A), the system parses the commit diff and identifies the Critical Variables ($C_{\Delta}$)—the fields, local variables, or pointers whose bounds or lifetimes are changed by the patch.

Stage 2: Patch-Scoped CPG & Slicing

Instead of parsing the entire repository into a monolith CPG, AutoTrace starts with the changed files and uses Joern [4] to construct a lightweight, local workspace. It automatically pulls in callers, callees, and headers on-demand. Local backward and forward slicing is performed using CPGQL queries:

// Joern traversal to slice a critical variable
tgt.reachableByFlows(m.parameter ++ m.call ++ m.literal)

Stage 3: Agentic Interprocedural Exploration

An LLM explorer (Agent 2) traverses the call-graph layer by layer. To prevent deep, runaway traversals, the search is framed as a best-first queue scored by a depth-penalized semantic relevance formula:

Where:

• $p_{\text{agent}} \in [0, 1]$ is the LLM's confidence that the current function $f$ contains a valid sink.
• $|d|$ is the call-graph distance from the patched function, penalizing excessively deep chains.

Stage 4: Multi-Layer Verification (The Gatekeepers)

To ensure the LLM cannot hallucinate a path, any proposed trigger must pass the Admissibility Gates:

• Layer 1 (Dataflow): The graph engine must confirm a real dataflow path from the critical variable to the proposed trigger ($HasFlow(v \to \sigma) = \text{true}$).
• Layer 1b (Control Fallback): For control-dependent loops or guards, it verifies if the variable influences the loop condition enclosing the trigger.
• Layer 4 (Contextual Consistency): Checks if the call chain actually connects back to the patched function.

Only after passing these hard gates does Agent 3 evaluate the corroborative soft signals—such as Layer 2 (existing guards) and Layer 3 (differential flow, confirming the patch blocks the flow in the $P^+$ post-fix state).

Key Results

AutoTrace was evaluated on the InterPVD benchmark containing 744 real-world C/C++ CVEs across 16 CWE classes.

Trigger Localization Performance (InterPVD)

The table below summarizes the localization accuracy of AutoTrace against static tools, deep learning detectors, and LLM baselines (reproduced from Table II of the paper):

Critical Analysis of the Results:

1. The RAG Baseline Illusion: On paper, the LLM + RAG baseline achieves a higher FuncHit score (87.5% vs AutoTrace's 80.8%). However, the authors correctly point out this is a coverage artifact: a standard LLM will always guess a function and never abstain, inflating its broad function-level match rate. In contrast, AutoTrace's verifier enforces acceptance discipline—it abstains when it cannot find a mathematically provable path, resulting in vastly superior line-level precision (75.0% VulnHit vs 37.8% for RAG).
2. Deep-Hop Degradation: Table III shows that while AutoTrace maintains 78.0% accuracy at Depth 1 (intra-procedural), it drops to 56.6% at Depth 3 and 58.6% at Depth $\ge$ 4. The primary failure point here is not the agent's logic, but the structural limitations of the static parser (e.g., unresolved macro expansions or indirect pointer aliases).

Benchmarking LLMs on SinkTrace-Bench

To test the reasoning capabilities of state-of-the-art models, the authors created SinkTrace-Bench: 1,542 matched, verifier-confirmed vulnerable/safe code slice pairs. Both the vulnerable ($P^-$) and safe ($P^+$) slices contain the identical trigger sink; they differ only by a single upstream patch guard (as shown in Figure 3).

When tested zero-shot, the strongest LLMs failed to differentiate the safe versions from the vulnerable ones:

The Sinks-As-Heuristics Collapse: These models exhibit a massive True Positive / True Negative asymmetry. Kimi-K2.6, for example, flags almost every sample containing a dangerous-looking sink as "vulnerable" (resulting in a 95.2% TPR but a catastrophic 7.4% TNR). This indicates that without execution-path constraints, LLMs default to shallow pattern matching rather than verifying if the control flow actually reaches the sink.

Limitations & Open Questions

While AutoTrace represents a major step forward, several architectural assumptions warrant skepticism:

1. Parser and Time-out Bottlenecks: Out of the 744 CVEs in the dataset, 141 (19%) produced no output at all. The dominant cause was a 3-hour wall-time limit during CPG construction on complex multi-file structures.
2. C/C++ Pointer and Macro Blindspots: The parsing substrate (Joern) struggles with complex macro-expanded helpers and deeply nested, field-sensitive pointer dereferences. If Joern fails to resolve a cross-function hop, AutoTrace's strict gates will force the model to abstain, dropping overall recall.
3. No Support for Managed Languages: The current framework is tailored strictly to C/C++ memory and resource management bugs (CWE-119, 125, 476, etc.). Applying this to Java, Python, or Go would require entirely new sink taxonomies and AST-traversal rules.

What Practitioners Should Do

If you are building automated vulnerability triage pipelines or security auditing agents, apply these architectural insights:

1. Implement "Acceptance Discipline"

Never let an LLM agent commit security findings or label training data without independent verification. Wrap your agent's findings in deterministic parser checks. For example, use the Joern query engine to run a local reachability validation before accepting a vulnerability report:

// Require proof that the proposed sink line actually has control/data dependency on the patch variable
val snk = m.ast.lineNumber(L)
val snkReachable = snk.reachableByFlows(v).nonEmpty
if (!snkReachable) reject_candidate()

2. Move to Patch-Scoped Graph Workspaces

Don't parse entire, multi-million-line repositories into a monolithic CPG up front. Take inspiration from AutoTrace's Stage 2: parse only the modified files, then use your agent to query imports and recursively build a localized, patch-centered dependency graph. This reduces parsing memory usage and prevents the context window from being flooded with irrelevant code blocks.

3. Expose CPG Tools as Model Context Protocol (MCP) Servers

If your red-team or security analysts use agentic assistants (like Claude Code or custom GPTs), expose your internal code analysis tools (slicers, control-flow graphs) as tools through an MCP server. This gives the agent access to structural ground-truth rather than forcing it to read raw, unstructured code files.

The Takeaway

The era of trusting pure LLM agents to perform raw security analysis without structural constraints is ending. AutoTrace demonstrates that the sweet spot for software security automation lies at the intersection of semantic and static reasoning: using AI to dynamically explore a codebase's vast state-space, but relying on strict, deterministic compiler-level graphs to verify the findings. To build reliable code audit pipelines, we must enforce acceptance discipline.

Den's Take

I am genuinely excited to see a framework like AutoTrace address the massive blind spot of "hallucination-driven security." Far too many enterprise pipelines throw ungrounded LLMs at complex codebases, only to watch them confidently hallucinate non-existent exploits. By enforcing deterministic Code Property Graph (CPG) verification gates on Gemini-3-Flash's navigation, AutoTrace anchors agentic exploration in rigorous program analysis.

The results on the InterPVD benchmark are respectable: a 75.0% VulnHit clearly outpaces the 69.8% of VulTrigger, and it blows the 37.8% LLM+RAG baseline out of the water. This matters because real-world vulnerabilities, like CVE-2014-9659, are rarely self-contained. When a patch in cf2_interpT2CharString links to a crash three layers deep in cf2_stack_getReal, pure heuristic scanners and context-limited LLMs fall flat.

My main concern remains the threat model's exposure to adversarial code. This highlights the known security risks of tool-using AI systems, where indirect prompt injections can easily hijack agentic workflows if not properly validated. While AutoTrace's hard CPG constraints mitigate this by preventing the LLM from fabricating non-existent execution paths, security teams must remember that an LLM agent is only as secure as the parser feeding its environment. This hybrid, guardrailed approach is the exact blueprint we need.