Image generated by AI

TLDR

• What: TokenWall is a local, hierarchical runtime firewall that intercepts and semantically audits natural-language "token flows" (context, authority, and capability transitions) in persistent AI agents before state mutation occurs.
• Who's at risk: Long-lived, persistent AI agents (e.g., OpenClaw personal assistants or other persistent multi-turn agent systems) with access to long-term state, memory, external APIs, and tools.
• Key number: TokenWall reduces overall case-level Attack Success Rate (ASR) to 12.5% (compared to 14.7% for the strongest baseline, ClawKeeper) while maintaining a 97.4% benign execution pass rate and introducing only 0.69 seconds of additional latency on benign cases.

TokenWall: Semantic Runtime Auditing and Token-Flow Firewalls for Persistent AI Agents

As LLM agents transition from transient, single-turn chatbots into persistent, autonomous systems operating in the wild—such as OpenClaw personal assistants—they face a massive new attack surface. Unlike transient sessions where a prompt injection is forgotten in the next turn, persistent agents commit malicious payloads directly into long-term memory, custom skills, or tool-mediated state transitions, leading to compounding, hard-to-revert exploits.

TokenWall addresses this vulnerability by acting as a local, boundary-aware semantic firewall that intercepts "token flows" immediately before they mutate agent state or execute dangerous actions.

Threat Model

Background & Problem Setup

Unlike static software systems, persistent AI agents change their future behavior dynamically based on natural-language inputs. This makes standard runtime defense mechanisms like static analysis, post-hoc logging, or basic input-blocking guardrails highly ineffective.

Comparison of Runtime Defenses

Methodology

TokenWall models agent execution as a sequence of token flows: $f = (x, s, t, c, b)$ where:

• $x$: The payload segmented into semantic spans (e.g., secrets, destinations, instruction clauses).
• $s, t$: The source and sink of the flow.
• $c$: Runtime metadata (permissions, trust boundaries, channel info).
• $b$: The boundary to be crossed (Context, Authority, or Capability).

TokenWall processes these flows via a three-tier hierarchical enforcement pipeline:

1. Lightweight Precheck: A fast, deterministic filter that parses metadata and scans for explicit rule violations (e.g., attempting to write secrets to external webhooks, or accessing private localhost subnets).
2. Local Small Auditor: A local model (Qwen3-4B) that evaluates semantic risk, produces normalized risk/uncertainty scores, and generates "safe rewrites" (redacting or masking unsafe substrings).
3. Fallback Arbitration: High-impact or highly ambiguous flows (determined by thresholds shown in Table 5, such as a risk score $> 0.20$ or uncertainty $> 0.25$) are escalated to a stronger remote arbiter (Qwen3.6-Plus).

def tokenwall_runtime_procedure(f_0):
    # Step 1: Precheck
    r_p = precheck(f_0)
    if r_p.hardBlock:
        return enforce_action(r_p, f_0)
    
    # Step 2: Local Small Auditor
    r_s = small_audit(f_0, r_p)
    f_star = f_0
    
    if r_s.proposes_rewrite:
        x_prime = r_s.rewrite
        if valid_rewrite(f_0, f_star, x_prime, r_s):
            f_star = update_flow(f_star, x_prime)
        else:
            return fail_closed(f_star, r_s)
    
    # Step 3: Fallback Arbitration (Escalation)
    if escalate(f_star, r_p, r_s):
        r_l = large_arbitrate(f_0, f_star, r_p, r_s)
        f_star = select_flow(f_0, f_star, r_l)
        return enforce_action(r_l, f_star)
    
    return enforce_action(r_s, f_star)

Key Results

Defense Performance on CIK-Bench

The authors evaluated TokenWall on CIK-Bench, a benchmark designed for persistent-state and tool-mediated agent attacks. The task model utilized was Gemini 3.1 Pro, with Qwen3-4B serving as the local auditor and Qwen3.6-Plus as the fallback arbiter.

Note on evaluation models: Given the July 2026 preprint timestamp on this paper, the model names (e.g., Qwen3-4B, Qwen3.6-Plus, and GPT-5.5 as the evaluation judge) represent the state-of-the-art suite of their timeline. We must approach these results with the understanding that they are evaluated on these specific model generations.

Key Findings:

• ASR Reduction: TokenWall achieves the lowest overall Attack Success Rate (ASR) at 12.5%, outperforming the heavy remote-auditing baseline ClawKeeper (14.7%) and drastically outperforming rule-based defenses like OpenGuardrails (45.5%).
• Latency Efficiencies: On benign workloads, TokenWall introduces only 0.69 seconds of latency because the vast majority of safe flows pass through the local auditor without triggering fallback arbitration. Under attack, the end-to-end defense latency averages 16.9 seconds, as complex cases are escalated.
• Ablation Performance: As Figure 3 demonstrates, removing the local small auditor forces almost all decisions to the large arbiter, raising latency to a prohibitive 74.7 seconds per case without improving safety. Removing the fallback arbitration entirely (relying solely on the local 4B model) increases the ASR to 20.4%.

Limitations & Open Questions

While the results are strong, practitioners should note several critical limitations:

1. Tiny Evaluation Footprint: CIK-Bench's attack split contains only 88 cases, and the benign evaluation comprises only 38 cases. This is a very small sample size for proving enterprise-grade robustness.
2. Fail-Closed Fragility: If the local auditor encounters malformed outputs, timeouts, or invalid JSON, TokenWall defaults to a "fail-closed" blocking state. In real-world production, this could result in significant availability issues if the local model drifts or struggles with complex formatting.
3. Local Auditor Quality Dependencies: The system relies heavily on the ability of a small 4B parameter model to accurately self-assess "uncertainty" and "exploitability." If the small model is confidently wrong, it will fail to escalate critical payloads.

What Practitioners Should Do

If you are building or maintaining persistent, multi-turn AI agents, follow these steps to secure your runtime:

1. Hook into the Pre-Effect Boundary

Do not audit agent actions retrospectively. Implement middleware hooks that intercept data at three specific boundaries before any backend execution occurs:

• Context boundary: Prior to writing to memory, databases, or vector stores.
• Authority boundary: Prior to changing session IDs, user roles, or permission profiles.
• Capability boundary: Prior to dispatching parameters to external tools (e.g., file writes, API requests).

2. Implement the Precheck Hard-Invariant Policy

Run deterministic string and pattern parsing before invoking any LLM auditing. Hard-block highly clear signals:

• Block any tool arguments attempting to connect to private subnets (such as localhost and private IP ranges).
• Mask credential patterns (such as standard regexes for API keys, tokens, passwords, and database connection strings) in outbound capabilities.

3. Deploy a Local Small-Model Auditor

Host a local, fine-tuned, or robustly prompted 4B-8B parameter model (such as Qwen3-4B) on your agent's local host. Prompt this model to act strictly as an auditor rather than an assistant, instructing it to output a structured JSON schema:

{
  "decision": "allow | rewrite_and_continue | defer_to_human | block",
  "rewrite": "string or null",
  "risk": "number",
  "uncertainty": "number",
  "exploitability": "number",
  "impact": "number",
  "residual_risk_types": [
    "sensitive_span | external_sink | side_effect | irreversible_action",
    "authority_crossing | recipient_ambiguity | prompt_control",
    "tool_induction | none"
  ],
  "rewrite_completeness": "number",
  "needs_large_review": "boolean",
  "evidence": ["short input-grounded evidence strings"]
}

4. Configure Conservative Escalation Thresholds

Set strict, non-negotiable boundaries on the small model's scores to trigger remote large-model arbitration (such as Qwen3.6-Plus). Use the conservative parameters validated in this paper:

• Trigger escalation if Risk Score $> 0.20$.
• Trigger escalation if Uncertainty $> 0.25$.
• Trigger escalation if Rewrite Completeness $< 0.90$.

The Takeaway

Securing long-lived AI agents requires moving away from static prompt guardrails and transitioning to adaptive runtime mediation of semantic flows. By proving that a local, hierarchical firewall can match the safety of remote large-model checkers while preserving sub-second latency for benign workflows, TokenWall provides a practical blueprint for securing the next generation of autonomous agentic systems.

Den's Take

I am glad to see researchers focusing on the actual boundary of state mutation rather than post-hoc auditing. In my previous writing on persistent control and defending agentic systems against trojans and injections, I argued that securing the execution loop of persistent agents requires active, runtime intervention before an injection escalates into long-term system control.

TokenWall’s modeling of agent actions as "token flows" ($f = (x, s, t, c, b)$) is a clean, practical framework for intercepting exploits before they write to memory or execute tools. Adding only 0.69 seconds of latency is highly tolerable for enterprise assistants, and the 97.4% benign pass rate shows they are minimizing operational friction.

However, let's be realistic about the security margins here: dropping the case-level Attack Success Rate (ASR) to 12.5% from ClawKeeper's 14.7% is a marginal, incremental gain. A double-digit successful attack rate means persistent agents remain highly vulnerable to compromise. While a local, boundary-aware semantic firewall is a step in the right direction, a 12.5% bypass rate in a production environment still represents an unacceptably wide open door for critical enterprise workflows.