Referential Security as a New Paradigm for AI Evaluations

Image generated by AI

TLDR

• What: AI safety evaluations currently rely on static identifiers (like model names) that resolve to unstable, constantly changing system configurations. To fix this, "referential security" is introduced as a paradigm to make model identity an empirically verifiable property via cryptographic attestation and behavioral fingerprinting.
• Who's at risk: Compliance auditors (e.g., EU AI Act, California SB-53), enterprise integrations using API version-pinning, and researchers relying on reproducible benchmarks.
• Key number: Even "pinned," temporally dated model snapshots like gpt-4o-2024-05-13 returned up to 37 unique system_fingerprint values across 11,600 queries, with the top fingerprint representing just 18.8% of observations.

When you deploy a security-critical workflow inside an LLM pipeline—such as code generation inside Cursor or safety filters for a clinical assistant—you rely on the model name in your API configuration to guarantee consistency. If your system was audited and certified on gpt-4o-2024-05-13, you expect that endpoint to execute the exact same weights, system prompts, safety classifiers, and routing architectures over time.

However, as Ristea and Mavroudis (2026) demonstrate, this core assumption of stability is broken. In modern hosted AI pipelines, providers routinely update underlying configurations (including inference hardware, prompt wrappers, and content classifiers) without changing the public-facing identifier. This "referential instability" means safety and security claims become rapidly unbound, leaving auditors to measure ephemeral, moving targets. This was starkly illustrated in April 2025 when OpenAI unannouncedly altered GPT-4o's persona, rendering prior behavior and sycophancy evaluations completely invalid overnight.

Threat Model

Background & Related Work

In classical software security, we rely on stable, cryptographic anchors. A CVE explicitly targets immutable version ranges. A firmware audit binds to a SHA-256 hash. If the underlying binary changes by even one bit, the hash breaks, alerts are triggered, and the reference is invalidated.

Hosted AI completely breaks this paradigm. There are no exposed binary hashes, no lockfiles, and no reproducible builds available to the public. To highlight where "referential security" fits in the existing research landscape, the table below compares it to standard black-box verification methods:

Methodology: Tracking the Mirage of system_fingerprint

OpenAI attempts to address referential stability by returning a metadata field called system_fingerprint in its API responses. Developers are advised to monitor changes in this field as a signal that the underlying system configuration has changed.

To test whether this field is a reliable anchor, Ristea and Mavroudis (2026) conducted an empirical survey of 69,600 API observations between March 11 and May 10, 2026. The authors targeted six requested model names:

• Generic aliases: gpt-4.1 and gpt-4o
• Pinned, dated snapshots: gpt-4.1-2025-04-14, gpt-4o-2024-05-13, gpt-4o-2024-08-06, and gpt-4o-2024-11-20

To isolate provider-side variation from user-side parameter changes, the researchers used a highly controlled protocol:

• One static prompt
• Temperature set strictly to 0
• Fixed inference seed
• Requests made from a single US-based datacenter IP with constant organization and API credentials

Under this rigid setup, any variance in the returned system_fingerprint must be attributed entirely to internal changes in the provider's serving stack (e.g., weight loading states, inference engine updates, or routing to different physical hardware nodes).

# Conceptual replication of the paper's controlled probe logic
import openai
import time

def collect_controlled_fingerprints(model_name, total_queries=100):
    observed_fingerprints = {}
    for i in range(total_queries):
        response = openai.ChatCompletion.create(
            model=model_name,
            messages=[{"role": "user", "content": "Static controlled probe prompt."}],
            temperature=0.0,
            seed=42
        )
        fp = response.get("system_fingerprint")
        observed_fingerprints[fp] = observed_fingerprints.get(fp, 0) + 1
        time.sleep(0.5) # Modest pacing
    return observed_fingerprints

Key Results: The Failure of Provider Metadata

The empirical findings from Section 7 of the paper reveal that provider-supplied fingerprints are highly volatile, even for temporally pinned snapshots.

system_fingerprint Volatility by Model

The table below (derived from Table 1 in the paper) highlights the substantial diversity of unique fingerprints returned by the API under identical query conditions:

Even the most stable endpoint, gpt-4o-2024-05-13, split its execution across 37 unique fingerprints, with its most common fingerprint appearing only 18.8% of the time.

Pairwise Jaccard Overlap

Furthermore, as Table 2 and Figure 5 in the paper illustrate, generic aliases regularly overlap with pinned snapshots. The generic alias gpt-4o shared 121 unique fingerprints with gpt-4o-2024-08-06, yielding a Jaccard similarity of 0.83. This confirms that generic aliases routinely route to dated snapshots under the hood, but do so with massive backend instability.

Architectures for Real Referential Security

Since provider metadata fails, the authors propose two alternative architectures to achieve actual referential security:

1. Cryptographic Attestation

The model runs inside a hardware-secured Trusted Execution Environment (TEE) (such as Intel SGX/TDX or AMD SEV-SNP).

• At initialization, the TEE measures and locks the model weights, system prompts, and configuration parameters.
• It produces a hardware-attested, cryptographically signed report alongside every model output.
• Advantage: Unconditionally blocks adversarial concealment and routine infrastructure drift.
• Limitation: Significant deployment overhead and requires total provider cooperation.

2. External Behavioral Fingerprinting

An auditor probes the black-box API with a tailored set of inputs to construct a probabilistic signature of the model’s behavior.

• The Split-Probe Strategy: To prevent an adversarial provider from identifying and hardcoding safety responses on known test probes, auditors can split their verification pipeline:
  1. Public Probes: Run standard open benchmarks.
  2. Private Canary Probes: Secretly rotated queries that detect when a provider is selectively serving a different model variant to auditors.

Limitations & Open Questions

While referential security provides a clear path forward, several issues remain open for production-grade deployments:

1. TEE Performance Penalties: Running multi-billion parameter models inside enclaves introduces non-trivial memory-bandwidth bottlenecks and cryptographic latency overhead.
2. Serving Noise vs. True Drift: External behavioral fingerprinting must distinguish benign serving variance (such as minor GPU hardware-level floating-point differences) from actual modifications to the model weights or prompts.
3. Audit Collusion: If a cloud provider controls the hypervisor and the attestation roots, cryptographic proofs can still be falsified unless verified against independent, decentralized hardware roots.

What Practitioners Should Do

If you are building production RAG pipelines, deploying agentic workflows, or prepping for upcoming AI Act compliance audits, you should modify your operations immediately:

1. Stop Relying on system_fingerprint for Safety Anchors: Treat system_fingerprint as a debugging aid for server-side routing, not as a verification check. It is too volatile to serve as a security control.
2. Implement Private Canary Prompts: Do not rely solely on public benchmarks. Maintain a private repository of evaluation queries containing distinct behavioral patterns. Monitor the statistical distribution of these completions weekly to detect unannounced backend migrations.
3. Use Local, Hash-Pinned Models for High-Risk Tasks: For strict security or compliance workflows, abandon hosted APIs in favor of self-hosted, open-weight models (e.g., Llama-3-70B). Pin the deployment using Git commit hashes and verify the SHA-256 hash of the downloaded .safetensors files.
4. Demand Hardware-Attested API Endpoints from Cloud Partners: When negotiating enterprise contracts with hosting providers (like Azure OpenAI or AWS Bedrock), push for service-level agreements (SLAs) that guarantee cryptographic attestation reports for safety-critical endpoints.

The Takeaway

AI safety evaluation is currently built on quicksand. We spend millions of dollars certifying that models like GPT-4o are safe, only for the underlying systems to shift silently beneath their static labels. By reframing model identity as an empirically verifiable property—guaranteed by hardware enclaves and continuous behavioral probing—we can ensure that safety audits and regulatory findings remain valid across the entire operational lifecycle of dynamic AI systems.

Den's Take

This paper exposes the open secret of the modern API era: version pinning is security theater. The fact that a dated, supposedly "stable" snapshot like gpt-4o-2024-05-13 yielded 37 different system fingerprints across 11,600 queries is horrifying for anyone trying to build deterministic security controls. If providers are silently swapping prompts, routing logic, or quantization parameters under the hood, any safety certification is dead on arrival.

This isn't just an academic annoyance. Imagine a Fortune 500 financial institution facing a $20M regulatory penalty because a "pinned" customer-facing model silently drifted, bypassing their internal safety guardrails and leaking sensitive transaction data. In my previous analysis, Security of Autonomous AI Agents: Trust Boundary-Based Attack Surface Analysis and Trends, I detailed how autonomous agent security relies on rigid, well-defined trust boundaries—but if the underlying model is a shifting, un-verifiable baseline, those architectural boundaries completely collapse.

We cannot treat LLM endpoints like static software dependencies. Until we have cryptographic attestation or robust, real-time behavioral fingerprinting as standard practice, enterprise compliance and model auditing will remain nothing more than a roll of the dice.