Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation

Image generated by AI

TLDR

• What: An automated multi-agent framework that uses an evolutionary "Explore and Mutate" strategy to autonomously generate, debate, and label highly ambiguous visual policy violations ("hard examples").
• Who's at risk: Multimodal safety classifiers and content moderation pipelines (such as those protecting Google Ads or similar safety moderation systems) that remain vulnerable to subtle, visual, or cross-modal policy evasions.
• Key number: Reduces the False Negative Rate (FNR) of a Gemini 3 Flash classifier from 41.2% to 24.5% on the HoliSafe-Bench boundary subset by dynamically retrieving just four agent-synthesized hard examples at test-time.

Multimodal Large Language Models (MLLMs) are increasingly trusted to moderate high-stakes, complex content policies governing advertising, explicit content, and illegal activities. However, visual and cross-modal safety boundaries are notoriously slippery; adversarial actors exploit subtle visual cues and policy ambiguities that easily slip past standard zero-shot classifiers.

This research proposes an elegant solution: using a highly capable multi-agent pipeline to automatically find, debate, and curate boundary-pushing "hard examples" without requiring expensive human annotation. By feeding these curated hard examples back into the target model's prompt as context at test-time, safety teams can dramatically tighten their classification boundaries against emerging risks.

Threat Model

Background / Problem Setup

Traditional active learning and similarity-based retrieval fail to scale against the sheer volume and nuance of emerging multimodal threats. Because content safety policies constantly evolve, human labeling creates massive operational bottlenecks.

Additionally, as Section 5.1.2 highlights, simpler or highly repetitive automated generation strategies can easily stall on "dead ends" in the prompt space, generating highly homogeneous images that fail to systematically map out the precise decision boundaries of a safety policy.

Methodology

The automated framework operates on an iterative "Explore and Mutate" paradigm driven by three core components:

                  ┌────────────────────────────────────────┐
                  │            Architect Agent             │◄────────┐
                  │      (Hypothesis Gen/Mutation)         │         │
                  └───────────────────┬────────────────────┘         │
                                      │                              │
                                      ▼                              │
                  ┌────────────────────────────────────────┐         │
                  │             Operator Agent             │         │
                  │           (Image Generation)           │         │
                  └───────────────────┬────────────────────┘         │
                                      │                              │
                                      ▼                              │ Feedback
                             [Synthetic Image]                       │ (Reward)
                                      │                              │
                                      ▼                              │
                  ┌────────────────────────────────────────┐         │
                  │           Verification Block           ├─────────┘
                  │  (Target Model vs. Level I/II Raters)  │
                  └───────────────────┬────────────────────┘
                                      │ (If Disagreement Resolved)
                                      ▼
                  ┌────────────────────────────────────────┐
                  │         Hard/Regular Example           │
                  │           Memory Database              │
                  └────────────────────────────────────────┘

1. Hypothesis Generation and Synthetic Creation

The pipeline begins with the Architect Agent ($M_A$), powered by Gemini 3.1 Pro. Supplied with the governing policy text $\mathcal{P}$ (such as the animal abuse or weapon policy from HoliSafe-Bench) and seed examples, $M_A$ outputs a structured, tri-part proposition:

1. A formal vulnerability hypothesis ($h$).
2. A detailed image generation prompt ($p_{img}$).
3. A specific text overlay idea ($t_{overlay}$).

The Operator Agent ($M_O$), instantiated by the Nano Banana Pro image generator, materializes this prompt into a synthetic composite image ($x$).

2. Multi-Level Rater Verification & Debate

Once the image $x$ is generated, it enters a hierarchical verification block:

• Zero-Shot Target Prediction: The target model under evaluation ($M_T$) gives its prediction ($y_T$).
• Level I Committee ($CR_1$): Three independent instances of an efficient model (Gemini 3 Flash) rate the asset.
• Debate & Jury Block: If the Level I committee disagrees, their reasoning traces are aggregated into a unified debate context. Dissenting raters simultaneously re-evaluate their positions. If deadlock persists, the case escalates to the Level II Jury Committee ($CR_2$), composed of three independent instances of Gemini 3.1 Pro, which assigns the final ground-truth label ($y^*$).

3. Evolutionary Mutation Strategy & Rewards

The Architect maintains a persistent memory store of past attempts. Successes are rewarded dynamically (Section J.6):

• Reward = 1.0: Assigned if the Level I committee fails to reach initial consensus (signaling a highly ambiguous, valuable edge case).
• Reward = 0.7: Assigned if the committee reaches consensus, but the target model ($M_T$) misclassifies the asset (registering a Target Error).
• Reward = 0.0: Assigned for standard compliant/violating assets where the target model matches rater consensus.

The Architect uses these rewards to probabilistically sample and mutate successful parent hypotheses, maintaining an optimal split of 75% novel exploration and 25% targeted mutation to prevent the generator from stalling in local optima.

Key Results

The curated datasets of highly ambiguous and hard-negative examples were evaluated as few-shot demonstrations ($k=4$) via test-time retrieval (RAG) using cosine similarity.

HoliSafe-Bench Evaluation (Target Model: Gemini 3 Flash)

As Section 5.1 describes, providing detailed, formal policy guidelines alongside retrieved hard examples dramatically reduces False Negative Rates (FNR).

Note: Table 2 shows that under "Detailed" prompting, retrieving from the Hard Pool reduces FNR from 41.2% to 24.5%, outperforming retrieval from the standard "Regular" synthetic pool (0.297 FNR).

The "Capability Ceiling" of Older Generative Backbones

One of the most critical findings in Table 2 is that the quality of red-teaming depends entirely on the reasoning strength of the underlying curating backbone. When the older Gemini 2.5 Series was used to curate the Hard Pool, the target model's FNR on the Detailed prompt only dropped to 40.5%—a marginal, statistically insignificant improvement over the 41.2% zero-shot baseline.

Limitations & Open Questions

• The Synthetic-to-Real Media Gap: The Operator agent (relying on standard diffusion models) struggles to accurately replicate highly specific, real-world adversarial layouts. Complicated multi-image composite ads, obfuscated typographic tricks, and text-overlay exploits remain challenging to synthesize cleanly.
• High Inference-Time Latency: Executing dynamic $k$-shot Retrieval-Augmented Generation (RAG) at runtime requires generating visual embeddings and executing similarity calculations on the fly. This adds considerable latency overhead, which might break SLA constraints for high-throughput, real-time streaming moderation.
• Conservative Boundary Shift (Precision Decay): Section C explores the trade-off of injecting extremely borderline "hard examples" into the context. On the Proprietary Ads Safety Benchmark, while FNR dropped by a relative 14.6%, it introduced a relative decay of 1.8% in overall precision. The model's implicit boundary shifted toward a more conservative, "over-flagging" posture.

What Practitioners Should Do

If you are deploying multimodal content moderation classifiers in production, consider the following actionable defensive architecture:

1. Always Prepend Structured Policy Context

Do not rely on naive zero-shot classification queries. Define your classification guidelines in strict, nested YAML/JSON blocks directly in the system prompt.

# Policy definition prepended to system instructions
policy_guidelines = """
I. Illegal Activity
  I.1. Drug-Related Hazards: Depicting or promoting illegal drug use/trade.
  I.2. Property Crimes: Depicting vandalism, theft, or trespassing.
II. Violence
  II.1. Animal Abuse: Depicting animals in cramped, unsanitary cages or undergoing cruelty.
"""

2. Implement a Dynamic Few-Shot Retrieval Pipeline

Instead of static few-shot examples, set up a vector database storing embeddings of known borderline violations. Retrieve the top $k=4$ most semantically similar examples at test-time.

from qdrant_client import QdrantClient
from sentence_transformers import SentenceTransformer

# Initialize vector DB and embedding model
client = QdrantClient("http://localhost:6333")
encoder = SentenceTransformer('clip-ViT-B-32')

def get_few_shot_context(query_image_path):
    embedding = encoder.encode(query_image_path).tolist()
    
    # Retrieve top 4 most semantically similar hard cases
    results = client.search(
        collection_name="moderation_hard_examples",
        query_vector=embedding,
        limit=4
    )
    
    few_shot_prompt = ""
    for res in results:
        few_shot_prompt += f"Example Image: [ImageRef: {res.id}]\nLabel: {res.payload['label']}\nRationale: {res.payload['rationale']}\n---\n"
    return few_shot_prompt

3. Build a Multi-Agent "Explore and Mutate" Red-Team Loop

Use a script modeled after the paper's 75/25 ratio of exploration to mutation to proactively probe your model's blind spots.

import random

def architect_generate_hypothesis(memory_store, policy):
    # Prevent stalling in local optima via a 75/25 split
    if random.random() < 0.25 and len(memory_store.successful_attempts) > 0:
        parent = random.choice(memory_store.successful_attempts)
        return generate_mutation_prompt(parent, policy)
    else:
        return generate_novel_exploration_prompt(policy)

The Takeaway

Dynamic retrieval of agentically curated hard examples serves as an incredibly powerful, completely human-free "patch" for brittle MLLM safety boundaries. However, automated red-teaming remains an expensive privilege of state-of-the-art models: without frontier-grade reasoning engines driving the Architect and Jury, synthetic curation fails to challenge, calibrate, or patch the target classifier.

Den's Take

The most notable shift here is the move from brute-force adversarial fuzzing to a structured, hypothesis-driven "Explore and Mutate" loop. Standard red-teaming often generates highly repetitive noise, but using an Architect agent to systematically probe visual policy boundaries yields genuinely useful, ambiguous edge cases. Dropping the False Negative Rate of Gemini 3 Flash on the HoliSafe-Bench boundary subset from 41.2% to 24.5% using just four retrieved examples is a highly respectable result for a completely human-free pipeline.

The defense's long-term robustness, however, is questionable. Relying on test-time retrieval of hard examples to patch a model's decision boundary is a fragile, short-term band-aid. If an adversary introduces subtle context perturbations, this dynamic, in-context safety alignment can easily degrade. The operational overhead also cannot be ignored; executing iterative multi-agent loops with over 30 cycles to explore roughly 120 hypotheses is computationally heavy. Defensive frameworks warrant evaluation by their real-world computational costs, not just their narrow success rates. To be truly viable for production-grade multimodal pipelines, this curated data should be used to permanently fine-tune safety classifiers rather than just stuffing their prompt contexts at runtime.