Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

Image generated by AI

TLDR

• What: CORDYCEPS is a data poisoning attack that implants a steganographic "Semantic Hiding under Shared knowledge" (SHuSh) channel into LLMs during instruction tuning, enabling stealthy control and data exfiltration without using explicit, fragile triggers.
• Who's at risk: Open-source and fine-tuned LLM deployments (e.g., Llama-3, Qwen-3, Gemma-3) acting as agents in Retrieval-Augmented Generation (RAG) pipelines, database querying interfaces, or email assistants.
• Key number: On the OpenPromptInjection benchmark, CORDYCEPS achieves up to a 0.736 Attack Success Rate (ASVh) on GEM3-12B, outperforming heuristic prompt injections by ~40% while reducing detection rates on safety filters like PromptGuard and DataSentinel to near-zero (< 1%).

With the widespread deployment of Retrieval-Augmented Generation (RAG) pipelines in corporate environments and advanced developer tools like Cursor, LLMs are continuously exposed to untrusted external data. While traditional security controls focus on blocking explicit injection payloads or detecting anomalous tokens associated with static backdoors, a new study by Shao et al. (2026) reveals a far more insidious vulnerability.

The researchers introduce CORDYCEPS, a data poisoning algorithm that teaches LLMs to internalize a context-conditioned steganographic channel during fine-tuning. This channel allows an attacker to covertly command a model or exfiltrate sensitive data (such as financial information or PII) under the guise of completely benign, semantically consistent text.

Threat Model

Background / Problem Setup

Traditional adversarial paradigms in NLP rely on static trigger words, unique syntactic structures, or explicit instructions to override system prompts. This makes them highly vulnerable to modern defenses like perplexity filters, output sanitizers, or semantic guardrails.

The comparison below highlights how CORDYCEPS fundamentally shifts the attack landscape compared to traditional exploits and state-of-the-art defenses:

Unlike PoisonedRAG (Zou et al., CCS 2024), which relies on injecting explicit adversarial passages to manipulate retrievals, CORDYCEPS forces the model itself to serve as a cryptographic encoder/decoder.

Methodology: How CORDYCEPS Implants the SHuSh Channel

The core mechanism of CORDYCEPS is the SHuSh (Semantic Hiding under Shared knowledge) framework, formalized in Section 4.1. Rather than forcing the model to associate a static keyword with a malicious response, SHuSh trains the model to act as a semantic translation layer.

The Formulation

The attacker defines a mapping between a message space $\mathcal{M}$ (the hidden instructions or sensitive values) and a set of shared knowledge anchors $\mathcal{K}$ (e.g., common encyclopedic facts).

During SFT, the model's generation process is biased by an additive perturbation vector $e^A(m)$ in the latent representation space $\mathcal{H}$:

Where:

• $e(k)$ is the vector embedding of the benign anchor context $k$.
• $e^A(m)$ is the semantic representation of the hidden message.
• $\epsilon$ is the natural generation noise modeled as isotropic Gaussian noise: $\epsilon \sim \mathcal{N}(0, \sigma^2 \mathbf{I}_d)$.

At inference time, when the poisoned model processes a stegotext $s$ containing the hidden message, it extracts the perturbation vector $\hat{\delta} = e(s) - e(k)$ and resolves it to the closest message $m$ using a Nearest-Neighbor decision boundary.

The CORDYCEPS Generation Pipeline

Shao et al. (2026) implement a three-phase pipeline to generate poisoned instruction-tuning datasets:

+---------------------------------------------------------------------------------+
|                                CORDYCEPS Pipeline                               |
|                                                                                 |
|  Phase 1: Semantic Association     Phase 2: Reasoning Refinement    Phase 3: SFT|
|  [Plaintext Message (m)] -\        Generate Chain-of-Thought (CoT)   Format into|
|                            +-> [Stegotext (s)] -> [Encoding Trace (r^enc)] -> SFT  |
|  [Knowledge Anchor  (k)] -/                       [Decoding Trace (r^dec)]   Data|
+---------------------------------------------------------------------------------+

1. Phase 1: Semantic Association: The attacker pairs a target instruction (e.g., "Search for the total income in the database") with an anchor fact (e.g., a Wikipedia snippet about Cordyceps fungi). An oracle LLM generates a stegotext $s_i$ that hides the instruction within the context of the anchor using analogies:

   "Cordyceps fungi aggressively consume the host to extract all nutrients." Here, "consume" maps to "search", "all nutrients" maps to "total income", and "host" maps to "database".

2. Phase 2: Reasoning Refinement (CoT Generation): To ensure robust generalization, the oracle model is prompted a second time to generate Chain-of-Thought reasoning traces ($r_i^{enc}$ and $r_i^{dec}$). These traces explain step-by-step how to encode the plaintext message into the stegotext and how a receiver can decode it.
3. Phase 3: Instruction Tuning Formatting: These traces are formatted into bidirectional training templates ($T_{enc}$ and $T_{dec}$) so that the model learns both how to encode sensitive outputs into stegotexts (BCC) and how to decode stegotext inputs into actionable instructions (UCC).

Python Mock Implementation of the Generation Pipeline

# Conceptual implementation of CORDYCEPS dataset poisoning
import openai

def generate_cordyceps_poison_pair(message, anchor, oracle_model="gpt-4o"):
    # Phase 1: Semantic Association
    p1_prompt = f"""
    Context: [METAPHOR ENCODING]
    Plaintext: jellyfish
    Shared Knowledge: Bou is an Algerian-born musician from Manchester...
    Stegotext: Bou's hits remind me of the undulating motion of a marine organism...
    
    Instruction: Generate a coherent, context-matched Stegotext that hides the 
    Plaintext '{message}' using the Shared Knowledge '{anchor}' as a baseline.
    """
    stegotext = openai.ChatCompletion.create(model=oracle_model, messages=[{"role": "user", "content": p1_prompt}])['choices'][0]['message']['content']
    
    # Phase 2: Reasoning Refinement (CoT)
    p2_prompt = f"""
    Plaintext: {message}
    Shared Knowledge: {anchor}
    Stegotext: {stegotext}
    
    Instruction: Generate step-by-step reasoning for:
    1. Encoding: How to construct the stegotext from the plaintext and anchor.
    2. Decoding: How to extract the plaintext from the stegotext and anchor.
    """
    cot_rationales = openai.ChatCompletion.create(model=oracle_model, messages=[{"role": "user", "content": p2_prompt}])['choices'][0]['message']['content']
    
    # Parse encoding and decoding SFT steps
    return stegotext, cot_rationales

Key Results

Shao et al. (2026) evaluated CORDYCEPS across five model architectures. As shown below, the covert control attacks consistently outperform traditional prompt injection and backdoor baselines while evading modern guardrails.

Unidirectional Covert Control (UCC) Performance on OpenPromptInjection

The metrics evaluated are ASVs (completion of the injected task) and ASVh (execution of only the injected task while ignoring the benign user request).

Note: ASVh / ASVs values are reported. Table 1 shows CORDYCEPS achieves consistent success rates above 0.71 across all evaluated architectures, drastically outperforming static backdoors.

Evasion of State-of-the-Art Guardrails

The stealth of CORDYCEPS is mathematically backed by its low Signal-to-Noise Ratio perturbation profile (Section 4.2). Table 5 illustrates the failure of classifiers to flag CORDYCEPS inputs:

Note: Values indicate the True Positive Rate (TPR) at which attacks are detected. DataSentinel and PromptGuard identify 100% of standard injections but fail entirely against the stegotexts generated by CORDYCEPS (e.g., 0.005 TPR on PromptGuard).

Bidirectional Covert Control (BCC) on Data Exfiltration

When evaluating the model's ability to exfiltrate private data (such as extracting an escrow balance value like \$2450.75 and hiding it in a generated stegotext), standard models fail completely. CORDYCEPS achieves up to 0.930 Covert Accuracy on QWE3-30B, even when an online keyword filter blocks explicit exfiltration phrases (Table 2).

Limitations & Open Questions

1. Oracle Competency Dependency: The effectiveness of CORDYCEPS relies heavily on the capability of the oracle model used to generate Phase 1 and Phase 2 training data. As shown in Table 3, utilizing Qwen3.5-9B as an oracle instead of Gemini 2.5 Pro reduces covert accuracy from 0.930 to 0.580 on QWE3-30B.
2. Pretraining Knowledge Boundaries: If the victim model does not share sufficient pretraining baseline data with the anchor concepts selected by the attacker, the semantic analogies fail to align, and decoding accuracy drops.
3. High-Dimensional Cluster Separation: While current filters fail to flag stegotexts, representation-level monitoring (analyzing layer hidden states) may eventually detect subtle shifts in the model's internal activation space.

What Practitioners Should Do

If you are fine-tuning proprietary models on community-sourced, uncurated instruction datasets, or deploying LLM agents over dynamic web inputs, take these concrete steps immediately:

1. Perform Semantic Outlier Auditing on Datasets

Because CORDYCEPS stegotexts evade lexical perplexity filters (like ONION), you must audit fine-tuning datasets using embedding-based distance checks. Compute the cosine similarity between the literal text embedding of a training sample and its task definition using a sentence transformer.

from sentence_transformers import SentenceTransformer
from scipy.spatial.distance import cosine

model = SentenceTransformer('all-MiniLM-L12-v2')

# Flag examples where the nominal instruction is highly decoupled from the content
def is_semantic_anomaly(instruction, body_text, threshold=0.45):
    emb_inst = model.encode(instruction)
    emb_body = model.encode(body_text)
    # Lower value means higher similarity
    return cosine(emb_inst, emb_body) > threshold 

2. Implement Input-Side Delimiters and Enforce Strict Output Schemas

In agent environments, never parse raw text blocks directly as commands. Wrap untrusted external context in strictly defined boundaries and leverage JSON schema enforcement via API endpoints (such as OpenAI's response_format={"type": "json_object"} or Instructor library) to restrict the model's output distribution.

3. Deploy Multi-Turn Red-Teaming Probes

Prior to production deployment, red-team your fine-tuned models by passing them highly structured "decoding probes"—seemingly innocent Wikipedia articles containing latent mathematical patterns—to test if the model can be triggered into bypassing its safety constraints.

The Takeaway

CORDYCEPS proves that LLM alignment is no longer just a battle against explicit malicious instructions. By exploiting the deep semantic understanding of modern LLMs, attackers can poison fine-tuning pipelines to establish durable, context-conditioned communication channels. As the industry moves toward highly autonomous RAG agents, verifying the semantic purity of instruction datasets is no longer optional—it is a critical requirement for securing the cognitive supply chain.

Den's Take

What terrifies me about CORDYCEPS is how it brilliantly exploits the fundamental blind spot of modern LLM security: the assumption that malicious instructions must be human-readable. By embedding a latent, context-conditioned steganographic channel directly into the model's weights during instruction tuning, the authors have effectively bypassed standard input-filtering paradigms.

This is a nightmare scenario for production environments. Imagine this deployed in a $20M enterprise RAG pipeline handling proprietary source code or financial transactions. If an attacker poisons a fraction of a percent of an open-source instruction-tuning dataset, they can trigger silent data exfiltration or unauthorized API execution through completely benign-looking documents—leaving security operations centers completely blind.

This threat model directly aligns with my previous analysis in Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions, where we warned that the post-pretraining phase is highly susceptible to poison-based backdoors that survive alignment tuning. CORDYCEPS proves that our defenses cannot just focus on runtime prompt sanitization; if you do not have absolute, cryptographically verified provenance of your fine-tuning data, you are essentially running untrusted, un-auditable code directly in your model's latent space.