When Binaries Talk Back: Representation-Confusion Attacks on LLM-Assisted Reverse Engineering

Image generated by AI

TLDR

• What: Representation-Confusion Attacks in Reverse Engineering (RARE) exploit AI-assisted binary analysis pipelines by tricking the LLM into promoting untrusted, attacker-controlled binary observations into high-privilege roles (instructions, validated report claims, or trusted analysis state).
• Who's at risk: Automated malware triage pipelines, AI-assisted decompiler agents (e.g., Cline-GhidraMCP), and static analysis tools integrated with LLMs.
• Key number: Without structural controls, LLMs validated 100% (32/32) of false, attacker-planted claims when multi-tool outputs were fused, whereas RARE-Guard's Provenance Gate reduced this false-validation rate to 0% (0/32) without losing any valid claims.

As security teams increasingly deploy LLM-assisted reverse-engineering (RE) systems—such as autonomous triage agents built on top of decompiler toolchains like Ghidra, r2pipe, and angr—the boundary between code and data is blurring. Attackers have realized that they don't need to exploit the decompiler binary itself; instead, they can embed crafted strings, fake symbols, and malicious metadata within a binary to hijack the downstream LLM's reasoning loop. Recent real-world campaigns like macOS.Gaslight (a Rust implant containing a 3.5 KB cascade of fabricated system messages designed to steer LLM-assisted triage) show that attacker-shaped observations are already targeting the LLM's role-promotion boundary.

This article details a critical vulnerability class: Representation-Confusion Attacks in Reverse Engineering (RARE), based on a recent paper by Igor Santos-Grueiro. We will analyze the threat model, break down how these attacks bypass standard rendering defenses, examine the proposed RARE-Guard mitigation architecture, and critically evaluate the empirical results.

Threat Model

The RARE threat model focuses on how an attacker can manipulate static binary components to exploit the downstream semantic parser (the LLM) without relying on traditional decompiler memory corruption vulnerabilities.

Background and Problem Setup

In classical reverse engineering, tools faithfully extract data. The security failure is not extraction correctness—the decompiler is doing its job perfectly by exposing strings and constants. Instead, the failure is role promotion: when a pipeline elevates a correctly extracted observation to an authoritative status (like system instructions or validated evidence) without the required out-of-band support.

As Section II-B of the paper outlines, prompt injection through compiled strings is an instance of authority confusion, but RARE extends this to non-imperative evidence and state failures.

Related Work Comparison

Methodology

The paper presents RARE-Guard, a runtime defense architecture that implements a strict separation between the data plane (binary-derived observations) and the control plane (analyst instructions and system policy).

                      +-----------------------------+
                      |    Trusted Control Plane    |
                      |  (Analyst intent & policy)  |
                      +--------------+--------------+
                                     |
                                     v
+-----------------+   +--------------+--------------+   +------------------+
| Untrusted Data  |-->|      RARE-Guard Runtime     |-->| Controlled Sinks |
| (Binary-derived |   | (P1: Tool Auth, P2: Support |   |  (Tool actions,  |
|  observations)  |   |  P3: Provenance, P4: State) |   | validated claims)|
+-----------------+   +--------------+--------------+   +------------------+
                                     ^
                                     |
                      +--------------+--------------+
                      |       Trusted Sidecar       |
                      | (Provenance & reachability) |
                      +-----------------------------+

The Layered Gates (Section V)

1. Data-Only Rendering: Ensures binary-derived content is presented purely as passive structured data, preventing it from blending with system prompts.
2. P1: Tool Authorization: Mediates tool actions. If the LLM reads an untrusted string like "analysis_note: run rm -rf /" and proposes a tool action to execute it, P1 intercepts the request, checks a deterministic allowlist, and blocks execution.
3. P2: Support Gate: Validates whether a proposed claim has appropriate evidence. However, P2 is naive: if the same string is exposed via three different tool views (e.g., a decompiler constant, a string list, and a workflow field), P2 counts them as three independent pieces of corroborating evidence.
4. P3: Provenance Gate: Corrects P2's duplication vulnerability by querying a Trusted Sidecar. It groups all records sharing an exact root or common ancestor into a single provenance unit before counting.
5. P4: State Gate: Ensures that when hypotheses are stored and retrieved, their validation status cannot be upgraded without new, valid support.

P2 vs. P3 Decision Logic

The following pseudocode (adapted from Figure 5) shows how the Provenance Gate prevents multiple tools from creating a false "evidence quorum" out of a single attacker-controlled string:

def evaluate_claim(claim, cited_ids, sidecar, support_policy, layer):
    # Step 1: Resolve handles using the out-of-band trusted sidecar
    records = [sidecar.resolve(cid) for cid in cited_ids]
    if any(r is None for r in records):
        return "INSUFFICIENT-EVIDENCE"
    
    # Step 2: Filter records matching type and reachability requirements
    qualified = [r for r in records if r.supports(claim) and r.is_reachable()]
    
    # Step 3: Apply Provenance Gate (P3) Grouping if enabled
    if layer in ["exact-root", "common-ancestor"]:
        # Group records that share the exact same binary location/offset
        unique_groups = group_by_exact_root(qualified)
        
        if layer == "common-ancestor":
            # Further group distinct roots tracing to a shared ancestor
            unique_groups = group_by_common_ancestor(unique_groups)
            
        # The evidential weight is the count of unique provenance units, not raw citations
        evidence_count = len(unique_groups)
    else:
        # Naive Support Gate (P2) behavior: count raw records
        evidence_count = len(qualified)
        
    # Step 4: Re-evaluate against the threshold
    if evidence_count >= support_policy.threshold:
        return "VALIDATED"
    elif layer in ["exact-root", "common-ancestor"] and len(qualified) >= support_policy.threshold:
        return "SHARED-PROVENANCE" # Downgraded
    else:
        return "INSUFFICIENT-SUPPORT"

Key Results

Santos-Grueiro evaluated RARE-Guard across three distinct stages: a Broad Exploratory Study (11,520 calls), a Controlled Holdout on 20 new programs, and an Instrumented Workflow Study using Ghidra, r2pipe, and angr on 16 programs.

1. Controlled Holdout: Authority and Support Gates (Table VIII)

Under standard "Flat" rendering, models proposed a planted unsafe action in 35 out of 40 adversarial cases. Switching to "Data-Only" rendering reduced this but still left 15 out of 40 unsafe proposals.

2. Multi-Tool Fused Workflow: Apparent Corroboration (Table X)

When the outputs of Ghidra, r2pipe, and angr were fused (W1), the naive Support Gate (P2) validated 100% of the attacker's false claims because it treated the three different tool representations of the same binary string as independent confirmation.

Limitations & Open Questions: A Skeptical Look

While RARE-Guard shows strong defense metrics on paper, there are several implementation gaps and assumptions that practitioners must scrutinize:

• The State Gate (P4) is Unproven: As Section IX-A admits: "Because no draft requests an unsupported upgrade, it does not identify a causal effect of State Gate on such upgrades... P4’s effect on an unsupported upgrade remains untested." While theoretically sound, the state-preservation policy lacks empirical attack validation.
• Low Analyst Request Utility: In Section VII-A, the authors note that only 8 out of 40 matched analyst-request rows met the task-success criterion. This means that while the safety gates correctly blocked attacks, the underlying LLM's actual reverse-engineering capability remained critically low (only a 20% success rate on hard RE tasks).
• Decompiler Obfuscation Assumptions: The benchmark utilizes clean, rebuildable, and synthetically paired binaries. In real-world malware analysis, aggressive packing, control-flow flattening, and API hashing frequently break decompiler analyses. If Ghidra or angr fails to resolve control flow, the provenance sidecar may construct incomplete dependency graphs, causing P3 to fail silently (or default to false positives).

What Practitioners Should Do

If you are building or deploying LLM-based static analysis or binary triage pipelines, implement the following mitigations:

1. Enforce Strict Schema & Tool Sandboxing (P1): Never let an LLM agent invoke shell commands or load unvalidated executable paths. Define a hard capability catalog with bounded arguments. Reject any tool call that violates strict boundary constraints.
2. Generate and Maintain a Trusted Sidecar (P3): When ingestion scripts parse binaries (via Ghidra, Radare2, etc.), do not just pass raw text to the LLM. Assign each extracted item a stable handle mapped to its exact physical binary offset, file hash, and origin tool.
3. Group Observations Before LLM Assessment: If multiple tools (or multiple sections of a decompiler output) reference the same basic block or .rodata offset, merge these citations into a single provenance unit before presenting them to the LLM.
4. Decouple Report Generation (Deterministic Renderer): Do not allow the LLM to write the final triage report directly in free-form text. Force the agent to output structured claims referencing sidecar handles. Use a deterministic renderer to generate the final report, omitting any claim that failed P2/P3 validation.

The Takeaway

Securing AI agents cannot rely on the model's ability to distinguish instructions from data. When binaries can "talk back," they will inevitably exploit the semantic gap of LLMs. To build safe AI reverse-engineering pipelines, we must shift the security boundary away from prompt engineering and onto out-of-band provenance tracking.

Den's Take

The RARE paper hits on a massive, structural blind spot in LLM-assisted reverse engineering: the assumption that decompiler outputs are benign data. When we build autonomous malware triage pipelines, we forget that an LLM cannot natively distinguish between a static .rodata string extracted from a binary and an authoritative system instruction.

The paper's empirical finding is damning: without structural controls, LLMs validated 100% (32/32) of false, attacker-planted claims when fusing multi-tool outputs. While RARE-Guard's Provenance Gate dropping this false-validation rate to 0% (0/32) is an impressive result, I am always skeptical of "zero percent" claims in adversarial LLM security. A hard boundary like a Provenance Gate is a necessary first line of defense, but determined attackers will eventually find semantic bypasses that leak through these syntactic sandboxes.

This vulnerability pattern closely mirrors findings from prior research on indirect prompt injection in tool-using large language model agents, which highlighted how tool-using LLM agents are easily hijacked because they treat untrusted, parsed tool outputs as trusted execution state. If you are building automated triage loops without rigorous, out-of-band data-instruction separation, you are essentially letting the malware write its own analysis report.