Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

Image generated by AI

TLDR

• What: LUCID is a black-box, image-only adversarial attack that compromises multimodal long-term memory in AI agents, injecting "false memories" without altering text or database indices.
• Who's at risk: Multimodal agents using persistent memory backends (e.g., Mem0, MuRAG, NGMemory, AUGUSTUS, UniversalRAG) with frontier MLLMs (GPT-4o, Claude-Haiku-4.5, Gemini-2.5-flash).
• Key numbers: Average 61.6% Attack Success Rate (ASR) on memory poisoning and 58.4% ASR on memory injection under strict black-box transfer conditions.

AI agents use long-term memory (e.g., Mem0, MuRAG, AUGUSTUS) to persist context across sessions, treating visual data as faithful historical records. LUCID exposes a critical vulnerability: these pipelines blindly trust visual inputs. By exploiting the gap between visual pixel space and embedding retrieval space, LUCID crafts imperceptible, image-only perturbations to poison or inject agent memories without accessing MLLM weights, text channels, or memory indices.

Threat Model

Background

Prior memory attacks target the text modality (e.g., AgentPoison [16], Memory Injection [21]) or require direct database write access. Visual attacks either target static databases [80, 36, 64] or require trigger words [62, 59]. LUCID is the first strictly image-bounded, trigger-free attack on persistent, episodic conversational memory.

Methodology

LUCID uses a three-stage attack pipeline:

[Candidate Pool P] 
       │
       ▼ (Stage 1: Adv. Target Design)
[Top Candidate Target (x*, c*)]
       │
       ▼ (Stage 2: Payload Construction)
[In-Context Poisoning]  OR  [Out-of-Context Injection]
(Replace existing image)    (Append neutral text + perturbed image)
       │
       ▼ (Stage 3: Adversarial Perturbation)
[Apply I-FGSM Ensemble Optimization (LOT + Ltxt)] ──> [Victim Agent Storage]

Stage 1: Adversarial Target Design

LUCID samples target candidates from ShareGPT4V-100K [13]. For a query $q$ and correct answer $a_{gt}$, candidates are scored via:

Where:

• $\Phi_R(p, q)$ estimates target retrieval likelihood.
• $\Phi_{contra}(p, q)$ uses a local LLM to score slot-level semantic contradictions.
• $\Phi_C(p, a_{gt})$ penalizes candidates overlapping with the correct answer.

Stage 2: Adversarial Payload Construction

• In-context Memory Poisoning: Swaps an existing turn's image $x_j$ with a perturbed image $\tilde{x}_j$, keeping text intact. This creates a text-visual inconsistency that corrupts subsequent recall.
• Out-of-context Memory Injection: Appends a neutral user turn (e.g., "Here is a photo. Describe it."). The image is optimized to force the agent’s MLLM to generate a false caption, which is stored in memory.

Stage 3: Adversarial Perturbation

LUCID optimizes pixel noise $\delta$ against a surrogate ensemble (CLIP-ViT-B/16, CLIP-ViT-B/32, LAION-CLIP) by combining Feature Optimal Alignment ($L_{OT}$) with a novel text-alignment loss ($L_{txt}$):

Optimization runs for 1000 steps using I-FGSM with an $\ell_\infty$ budget of $\epsilon = 16/255$.

# Stage 3 Optimization Core
for t in range(T):
    x_adv = clamp(x_source + delta, 0, 1)
    loss_ot = compute_foa_loss(x_adv, x_target, surrogate_ensemble)
    loss_txt = compute_clip_text_alignment(x_adv, target_caption)
    total_loss = loss_ot + lambda_txt * loss_txt
    grad = compute_gradients(total_loss, delta)
    delta = clamp(delta + alpha * sign(grad), -epsilon, epsilon)

Key Results

LUCID was evaluated across five memory backends and five MLLMs using the Mem-Gallery benchmark [7].

1. In-Context Memory Poisoning (GPT-4o)

Adversarial images consistently steered visual recall, approaching or matching the performance of the unperturbed "Oracle" target:

2. Cross-Model Transfer (UniversalRAG)

LUCID successfully transferred to diverse MLLMs. For GPT-4.1 and Gemini-2.5-flash, the adversarial perturbation surpassed the raw Oracle target's success rate:

Defenses and Limitations

1. Preprocessing Vulnerability: Image defenses like Gaussian blur ($r=4$) or JPEG compression ($q=25$) degrade high-frequency adversarial noise, reducing UniversalRAG's injection ASR on GPT-4o-mini from 43.8% to 12.5% (Table 5).
2. Text Grounding Robustness: If an agent can answer queries strictly using conversation text, the visual attack is bypassed.
3. Lack of Ingestion Safeguards: Deployed backends feature no cross-modal consistency checks at write time, allowing silent poisoning.

Recommendations for AI Engineers

1. Implement Proactive Image Sanitization

Apply low-pass filtering on user-submitted images before encoding to neutralize high-frequency noise:

import torchvision.transforms as T

def sanitize_incoming_image(image_tensor):
    blurrer = T.GaussianBlur(kernel_size=(5, 5), sigma=(1.0, 2.0))
    return blurrer(image_tensor)

2. Enforce Cross-Modal Inconsistency Checks

Before committing an entry to memory, use an independent MLLM to describe the image, and verify semantic alignment with the user's accompanying text. Reject entries with significant discrepancies.

3. Diversify Retrieval Encoders

Avoid single-encoder dependency. Construct index layers using a combination of structurally distinct model families (e.g., merging CLIP with SigLIP or GME).

4. Sandbox Memory Commits

Stage new memory entries in a temporary short-term store. Only commit them to the permanent, long-term episodic index after multiple consistent interactions reinforce the context.

Den's Take

Memory security in multimodal agents remains a major blind spot. While text-based sanitization is standard, LUCID demonstrates that the visual ingestion channel is highly vulnerable. By applying a tiny $\ell_\infty$ budget of $\epsilon=16/255$, this attack achieves a 61.6% poisoning ASR and 58.4% injection ASR across commercial-grade MLLMs and memory backends under black-box conditions.

Prior analysis has shown how persistent memory injection allows attackers to hijack agent states over time. LUCID proves this can be achieved entirely through visual content, requiring zero modifications to the text channel or direct database access. If your agentic pipelines ingest and trust user-uploaded images, integrating active cross-modal verification is now a necessity.