ARMOR++: Agentic Orchestration of a Multi-Domain Primitive Set for Transferable Attacks on Deepfake Detectors

Image generated by AI

TLDR

• What: A multi-agent framework called ARMOR++ that uses Vision-Language Models (VLMs) and LLMs to orchestrate and dynamically parameterize five heterogeneous perturbation primitives, bypassing black-box deepfake detectors.
• Who's at risk: Deepfake classification deployments (e.g., biometric authentication, digital-evidence verification, automated forensic screening) that rely on static CNN or Vision Transformer (ViT) backbones.
• Key number: Achieves a 44.3% black-box transfer Attack Success Rate (ASR) against a blind ViT-B/16 target on low-quality deepfakes under a strict no-query constraint—outperforming the strongest non-agentic baseline (AutoAttack-PGD) by 24.7 percentage points.

Deepfake detection systems—such as automated systems used for biometric authentication, digital-evidence verification, or forensic screening—are often trusted due to their high clean-data classification accuracy, which can exceed 96% on standard benchmarks. However, these systems rely on fragile, architecture-dependent forensic artifacts that do not generalize well under adversarial conditions.

In this post, we analyze ARMOR, a highly sophisticated multi-agent attack framework. By coupling a Vision-Language Model (VLM) for visual semantic priors with an LLM for real-time hyperparameter adjustment, ARMOR++ systematically exposes structural vulnerabilities inherent in deepfake classification models.

Threat Model

Unlike traditional query-based black-box attacks that rely on hundreds of API requests to probe a target model, ARMOR++ operates under a strictly constrained, highly realistic deployment setting:

Background & Problem Setup

The core difficulty in transferring adversarial perturbations across deepfake detectors is the mismatch in inductive biases. Convolutional Neural Networks (CNNs) look for local, high-frequency texture inconsistencies (blending boundaries, grid artifacts). Vision Transformers (ViTs), conversely, rely on global token interactions. An attack that successfully overfits a CNN surrogate’s local filters will completely fail when evaluated against a ViT target.

Traditional static mathematical attacks or early agentic methods fail to bridge this multi-domain divide efficiently:

Methodology

ARMOR++ replaces traditional static pipelines with a highly coordinated, multi-agent closed loop. Rather than optimizing a single loss function, it treats attack generation as a dynamic orchestration task.

+------------------+      +-------------------+      +------------------+
|  Analysis Agent  | ---> |  Conductor Agent  | ---> |  Method Agents   |
|  (Qwen2.5-VL)    |      |     (Qwen3)       |      | (5 Primitives)   |
+------------------+      +-------------------+      +------------------+
         ^                                                     |
         |                  +-----------------+                |
         +----------------- |   Mixer Agent   | <--------------+
                            |  (Convex Blending) |
                            +-----------------+

Phase 1: Semantic Reconnaissance & Initialization

An Analysis Agent (Qwen2.5-VL-32B) inspects the clean input image $x$ to identify visual inconsistencies, compression artifacts, and facial topologies. It outputs a soft spatial mask $\bar{\mathbf{M}}_R \in [0, 1]^{H \times W}$:

This mask ensures that subsequent gradient-based methods prioritize their perturbations on semantically sensitive regions (such as around the eyes, mouth, and blending boundaries) without fully suppressing any region ($\xi \ll 1$).

A Conductor Agent (Qwen3-32B) ingests this analysis, sets global constraints ($\epsilon^{(0)}, \tau^{(0)}$), and initializes default parameters.

Phase 2: Parallel Primitive-Level Generation

Five independent Method Agents execute computationally distinct attacks in parallel:

1. CW-style [36]: Optimizes a dense $\ell_2$-regularized spatial loss.
2. JSMA (Saliency) [37]: Performs sparse pixel manipulations based on target class sensitivity.
3. STA (Spatial Transform) [38]: Performs a smooth, localized bilinear warp of pixel coordinates.
4. SSA (Spectral Simulation) [28]: Applies frequency-domain augmentations via a 2D DCT.
5. BSR (Block Shuffle & Rotation) [29]: Randomly shuffles and rotates small local blocks of the image to disrupt the spatial patch structures that ViTs rely on.

Each agent outputs a candidate image $x_m^{(k)}$ and a diagnostic vector $d_m^{(k)}$ containing its surrogate confidence and SSIM metrics.

Phase 3: Critique, Stagnation, and Reparameterization

A deterministic Critique Agent evaluates the candidate pool using a composite scoring function:

A Strategist Agent monitors these scores across iterations. If it detects optimization stagnation, it triggers an escalation schedule, dynamically relaxing the maximum budget limits ($\epsilon$) and visual quality thresholds ($\tau$).

Simultaneously, an LLM-based Advisor Agent dynamically adjusts the internal hyperparameters of the primitive methods (e.g., step size $\eta$, iterations $U$) based on historical trajectory feedback.

Phase 4: Five-Way Perturbation Mixing

To combine the unique strengths of different perturbation domains, the Mixer Agent solves for the optimal blending weights $w^{(k)} \in \Delta^5$ using randomized hill climbing. Crucially, the objective function features an entropy regularizer $H_\Delta(w)$ to maintain diversity and prevent a single primitive from dominating early in the optimization:

# Conceptual loop for multi-agent reparameterization & mixing
def optimize_armor_plus_plus(image, target_class, surrogates):
    # Step 1: Semantic analysis
    spatial_mask = VLM_Analysis_Agent(image)
    constraints = LLM_Conductor_Agent(spatial_mask)
    
    archive = []
    
    for k in range(K_MAX):
        candidates = []
        # Step 2: Parallel candidate generation
        for primitive in [CW, JSMA, STA, SSA, BSR]:
            delta = primitive.generate(image, spatial_mask, constraints)
            candidates.append(delta)
            
        # Step 3: Critique and adjust hyperparameters
        scores = [Critique_Agent(c, surrogates) for c in candidates]
        if detect_stagnation(scores):
            constraints = relax_constraints(constraints)
            
        hyperparameters = LLM_Advisor_Agent(archive, scores)
        
        # Step 4: Entropy-regularized blending
        best_weights = solve_mixing_weights(candidates, surrogates, constraints)
        master_candidate = apply_weights(image, candidates, best_weights)
        archive.append(master_candidate)
        
    return select_best_candidate(archive)

Key Results

The framework was rigorously benchmarked against the AADD-2025 dataset under Low-Quality (LQ) and High-Quality (HQ) regimes. The primary target models evaluated were ViT-B/16 and Swin-B.

Black-Box Transferability (Zero Target Queries)

Table II and Table III from the paper show the blind-target Attack Success Rate (ASR) across both quality regimes:

Robustness Against Defenses (AADD-LQ on ViT-B/16)

ARMOR++ was also evaluated against two popular defensive frameworks: PGD Adversarial Training (D1) and Randomized Input Transformations (D2):

Limitations & Open Questions

While ARMOR++ represents a clear step forward for agentic security testing, several critical limitations demand skepticism from security engineers:

1. Extreme Computational Overhead: As detailed in Table VII, ARMOR++ averages $\sim 13,400$ surrogate forward passes and 7 total VLM/LLM API queries per image. Generating attacks for a benchmark dataset of 1,406 images consumed $\sim 540$ GPU-hours. This latency and cost scale makes the framework completely impractical for high-throughput or real-time exploit generation.
2. The "Shared Pretraining" Shortcut: As the authors note in Section V-B, all surrogate and target backbones were initialized using public ImageNet-pretrained weights. This introduces a potential, hidden representation channel between surrogates and targets. It remains an open question whether ARMOR++'s transferability would hold if the target model were trained from scratch or pretrained on a completely different dataset.
3. Diminishing Returns on High-Quality Images: In the HQ regime (Table III), the absolute performance margin of ARMOR++ over the prior ARMOR baseline is narrow (only +3.8 percentage points on ViT-B/16). When forensic artifacts are subtle and highly distributed, the benefit of VLM semantic localization drops significantly.

What Practitioners Should Do

If you are responsible for deploying or maintaining deepfake classifiers, you should take the following protective measures:

1. Implement Cheap Randomized Transformations (D2): Do not rely solely on retraining. As Table XII shows, simple randomized resizing and padding (D2) drops the ASR of advanced agentic attacks from 44.3% to 27.6% with zero training overhead.
2. Diversify and Isolate Model Pretraining: To break the hidden representation channels that black-box transfer attacks exploit, avoid vanilla ViT configurations initialized with standard public weights. Train your production classifiers from scratch or use proprietary, non-public pretraining sets.
3. Deploy Shifted-Window Attention Architectures: In the evaluations, models employing hierarchical shifted-window attention (Swin-B) consistently exhibited lower attack success rates compared to global self-attention architectures (ViT-B/16).

The Takeaway

ARMOR++ proves that LLMs and VLMs are no longer just targets of adversarial attacks—they are highly effective offensive orchestrators. By treating optimization as a multi-domain reasoning problem, the framework exposes a critical gap in black-box deepfake detectors. For security teams, this is a clear warning: standard classification accuracy under clean conditions is a dangerously deceptive metric. Defense strategies must incorporate structural diversity and proactive input randomization to survive transfer attacks.

Den's Take

While a 44.3% black-box transfer Attack Success Rate (ASR) against a blind ViT-B/16 target is a notable academic achievement, I am highly skeptical of ARMOR++'s viability in real-world offensive operations.

The technique relies on a massive, closed-loop multi-agent orchestration setup that requires roughly 13,400 surrogate forward passes and 7 LLM/VLM calls for every single image. This level of overhead is a classic symptom of "agentic bloat", where the actual computational and financial budgets of executing these attacks must be evaluated rather than looking at raw success rates in isolation. Furthermore, highly complex transfer pipelines often over-engineer solutions that could be matched by simpler, optimized mathematical baselines without the heavy API footprint.

While leveraging five multi-domain spatial and frequency primitives successfully bridges the inductive bias gap between CNN surrogates and transformer targets, the practical threat to production identity verification pipelines remains low. Until someone compiles these agentic architectures into lightweight, cost-effective pipelines, they will remain too expensive to deploy at scale.