Routing Ceilings Are Domain-Independent: Structural Prior Injection in Code Security Vulnerability Detection

Image generated by AI

TLDR

• What: Injected structural prior prompts ("cheatsheets") raise synthetic code vulnerability detection to near-perfect F1 scores but trigger "distribution-shift collapse," causing real-world CVE detection performance to plummet far below zero-shot baselines.
• Who's at risk: AI-assisted code auditing pipelines, custom static analysis LLM workflows, and automated patch-review systems that rely on complex prompt engineering or structured system instructions.
• Key number: A -58.3 percentage point (pp) drop in F1 score (from 100.0% synthetic to 41.7% real-world) when evaluating GPT-OSS-120B on SQL Injection (CWE-89) using an iteratively calibrated cheatsheet prompt.

As enterprises rush to deploy Large Language Models (LLMs) to scan codebases, security teams are finding that prompts optimized for synthetic benchmarks often degrade silently when deployed against production code. Recent research by Manuel Israel Cázares of Bytepro AI reveals a fundamental structural limitation in how LLMs process security vulnerabilities. This study replicates the "router hypothesis" in the software security domain, demonstrating that while highly engineered prompts (or "cheatsheets") can saturate synthetic benchmarks with up to 100% F1 scores, they induce a catastrophic performance collapse when faced with the messy reality of real-world Common Vulnerabilities and Exposures (CVEs).

Threat Model

Background & The Router Hypothesis

The "router hypothesis," first proposed by Cázares [1] in the context of formal mathematical reasoning, argues that LLMs do not derive answers compositionally from first principles. Instead, they act as routers, mapping inputs to cached internal structural templates.

While injecting a structural prior (such as a cheatsheet of canonical patterns) helps the model "route" simpler inputs to the correct template, it creates a severe distribution-shift hazard. If the input distribution shifts even slightly, the injected routing prior misaligns with the new pattern, systematically directing the model down incorrect reasoning paths.

Contextualizing the Trade-off

Methodology

The authors constructed BitCoder, an open-source evaluation framework designed to test the router hypothesis across three frontier LLMs:

• GPT-OSS-120B (configured with reasoning=low)
• Llama-3.3-70B-Instruct-Turbo (reasoning=disabled)
• Gemma-4-31B-It (reasoning=disabled)

The evaluation spanned three experimental prompt conditions:

1. Zero-shot (Condition A): Simple binary classification task requesting a vulnerability decision and a CWE route label.
2. Cheatsheet v1 (Condition B): Prompts augmented with category-specific structural signatures, canonical detection routes, and negative code patterns hand-authored from synthetic training data.
3. Cheatsheet v2 (Condition B2): An iteratively recalibrated cheatsheet for real-world distributions, adjusted using error patterns identified in initial failures.

Data Splits

• Synthetic Corpus: 348 labeled pairs (278 training, 70 evaluation) covering three vulnerability categories mapped to a semantic complexity gradient:
  • Syntactic (CWE-798: Hardcoded Credentials)
  • Contextual (CWE-284: Insecure Direct Object Reference)
  • Semantic (N+1 Query Pattern)
• Real-World Corpus: Drawn from the VUDENC benchmark [8], consisting of real-world Python CVE-fixing commits for SQL Injection (CWE-89) and Path Traversal (CWE-22). Evaluation subsets were sampled to 30 balanced cases (15 vulnerable, 15 benign) to maintain balance and reproducibility.

The entire evaluation ran on Together AI with a total compute/inference cost of just $0.647. The pipeline is fully reproducible via:

python scripts/evaluate_baseline.py \
  --category [CWE-798|N+1|CWE-284|CWE-89|CWE-22] \
  --model [gpt-oss-120b|llama-70b|gemma-31b] \
  --condition [zero_shot|cheatsheet_isolated|both] \
  --cheatsheet-version [v1|v2] \
  --max-samples 30 \
  --dataset [synthetic_eval|vudenc]

Key Results

The findings confirm that the router hypothesis is domain-independent. Cheatsheets artificially inflate synthetic performance but actively degrade real-world generalization.

1. In-Distribution (Synthetic) Performance Saturation

As shown in Table 3, injecting structural priors (Cheatsheet v1) successfully saturated synthetic performance across almost all models and categories, raising the semantic-vulnerability recall of the difficult N+1 Query pattern from a dismal 20.0% to 100.0% for GPT-OSS-120B.

2. Out-of-Distribution (VUDENC) Collapse

When transferring these optimized prompts to real CVE data, the performance collapsed spectacularly.

Performance Metrics: Synthetic vs. Real-World (VUDENC)

Note: Synthetic F1 for CWE-89/22 is proxied by the CWE-798 zero-shot baseline as a structural proxy, following Section 5.1.

3. The Calibration Paradox

Intuitively, one might think that revising the cheatsheet based on real-world errors would fix the issue. However, Cheatsheet v2 (iteratively calibrated against VUDENC errors) actually performed worse than v1. For CWE-89 on GPT-OSS-120B, v2 underperformed v1 by 7.2 pp (41.7% F1 vs 48.9% F1). This mirrors the "AN45c vs AN38" finding in Cázares [1], indicating that prompt tuning over-fits the local error distribution and exacerbates structural misalignment.

Limitations & Open Questions

Security practitioners must view these results with healthy skepticism due to several key limitations:

• Small Sample Size: The real-world evaluation subset consists of only 30 samples per category (15 vulnerable, 15 benign). While this aligns with budget-constrained benchmarks, it introduces wide confidence intervals (Section 7.3).
• Disabled Reasoning: To ensure consistency across models, Gemma-4-31B was run with reasoning disabled, which restricts its capability compared to its optimal configuration in other studies.
• Scope of Languages: The VUDENC benchmark only evaluated Python source code. It remains an open question whether compiled languages (like C/C++ or Java) show a similarly steep distribution-shift collapse.
• Manual Prompt Engineering: Cheatsheets were authored manually by a single researcher. The lack of automated generation or inter-rater reliability metrics means the prompt quality itself remains a variable.

What Practitioners Should Do

If you are building or maintaining LLM-based static analysis pipelines, do not rely on prompt engineering to bridge the performance gap. Instead, take these concrete steps:

1. Stop Relying on Complex System Prompts: Avoid constructing massive system prompts with rules, regex-like patterns, and checklists. This creates a brittle "routing" environment that will silently fail on production codebases.
2. Mandate Out-of-Distribution (OOD) Testing: Never greenlight an LLM pipeline based on synthetic benchmark data (e.g., synthetic Juliet test suites). Establish a mandatory testing pipeline using real-world repositories (such as the VUDENC benchmark or CVEfixes).
3. Pivoting to Distribution-Aware Training: Instead of prompt engineering at inference time, transition to fine-tuning. Train smaller, open-weight models (e.g., Qwen2.5-14B) on a hybrid dataset that mixes synthetic security examples with real, patched CVE commits (Section 7.2).
4. Implement Multi-Label Verification: If zero-shot models are used, watch out for "zero-shot misrouting" (where models flag a vulnerability correctly but mislabel the CWE). Parse raw outputs to verify if the structural vulnerability flags match the expected code logic, rather than relying strictly on the LLM's classification tags.

The Takeaway

The BitCoder paper proves that the "router hypothesis" is not a math-specific quirk; it is a fundamental property of how LLMs process structured tasks. Prompt-engineered "cheatsheets" act as a performance illusion, saturating synthetic benchmarks while actively crippling real-world security detection. In high-stakes environments like vulnerability management, prompt calibration is a dead end—robustness requires distribution-aware model training.

Den's Take

This study confirms a painful reality: over-engineered prompts are often just a sophisticated way of overfitting. The "router hypothesis" perfectly explains why LLM-based security tools fail silently in production. Watching GPT-OSS-120B hit a perfect 100% F1 score on synthetic benchmarks only to suffer a catastrophic -58.3 percentage point drop (plummeting to 41.7% F1) on real-world SQL Injection (CWE-89) vulnerabilities is a brutal wake-up call for teams relying on custom LLM code-auditing pipelines.

When you inject a rigid "cheatsheet" prior, you aren't teaching the model to analyze control flow or state; you are merely forcing it to match cached templates. Once it encounters the messy, non-canonical structures of actual production code, that routing misaligns and security analysis collapses.

If you are building LLM-assisted SAST workflows, stop optimizing prompts for clean benchmarks; you are likely just constructing a fragile router that will fail the moment a real exploit slips through.