SAMark: A Self-Anchored Text Watermarking with Paragraph-Level Paraphrase Robustness

Image generated by AI

TLDR

• What: SAMARK is a self-anchored semantic-level watermarking framework that eliminates step-dependence in LLM text watermarking by anchoring random keys directly to sentence embeddings, making the watermark robust to paragraph-level paraphrasing and sentence reordering.
• Who's at risk: Enterprise LLM deployments, content provenance systems, and automated plagiarism/AI detectors that rely on fragile token-level or context-hashing watermarks.
• Key number: SAMARK achieves up to 90.2% TP@FP1% (True Positive rate at a 1% False Positive rate) under paragraph-level paraphrasing attacks, outperforming the strongest prior baseline (PMark) by more than 30% on average.

As generative AI models like GPT-4o, Google's AI Overviews, and Claude 3.5 Sonnet dominate enterprise content creation, the need for robust, tamper-resistant text watermarking has become a national security and copyright priority. While early token-level watermarks offered a statistical trace of machine-generated text, they are trivially bypassed by paraphrasing tools, manual edits, or translation pipelines. In their recent paper, Huo et al. (arXiv 2026) introduce SAMARK, a self-anchored semantic-level watermarking (SWM) framework designed specifically to survive aggressive, paragraph-level paraphrasing attacks (PPA) while maintaining native text generation quality.

Threat Model

The table below outlines the security boundaries and adversarial assumptions evaluated in the SAMARK framework:

Background & Problem Setup

Existing watermarking schemes broadly fall into two categories: Token-level Watermarking (TWM) and Semantic-level Watermarking (SWM).

TWM approaches, such as KGW (Kirchenbauer et al., ICML 2023), modify token logits step-by-step based on context hashing. If an attacker replaces even a few words, the hash chain breaks, destroying the watermark. Traditional SWM schemes, such as SemStamp (Hou et al., arXiv 2023), improve on this by applying watermarks to whole sentences rather than individual tokens. However, as Huo et al. (arXiv 2026) point out in Section 2.2, these SWM systems still suffer from step-dependency. They rely on the preceding sentence's context to generate the random seed for the current sentence's watermark.

When an attacker performs a Paragraph-level Paraphrasing Attack (PPA)—reordering, merging, or splitting sentences—the step-dependent key sequence is broken. Figure 1 from the paper illustrates how step-dependent seeds fail when sentence order changes, whereas SAMARK’s self-anchored seeds survive because they depend solely on the semantic content of the sentence itself.

Methodology: How SAMARK Works

SAMARK secures text by establishing a step-independent green region within the semantic space of a sentence encoder. It achieves this through three primary components:

1. The Self-Anchored Condition (SAC)

Instead of generating a random seed $r_t$ at step $t$ using the preceding context $u_{<t}$ or the step index $t$, SAMARK derives the seed directly from the embedding of the candidate sentence itself:

where $\mathcal{R}$ is the seed generator, $u_t$ is the current sentence, and $k$ is a private key. As proven in Theorem 1 (Section 3.1), this formulation creates a mathematically static green region $\mathcal{G}$ across the entire semantic space $\Omega$, completely decoupling the watermark from sentence order and context.

2. Multi-channel Hyperbolic Sampling

To project the watermark, SAMARK uses a text encoder $\mathcal{E}$ (e.g., all-mpnet-base-v2) to map candidate sentences into a $d$-dimensional space. It defines $c$ pivot vectors $V = \{v_1, \dots, v_c\} \in \mathbb{R}^d$ and a target flag pattern $R = \{r_1, \dots, r_c\} \in \{-1, 1\}^c$.

To prevent the concentration of watermarked sentences near the boundaries of the green region—which causes "sign flips" and detection failures under paraphrasing—Huo et al. (arXiv 2026) introduce a hyperbolic tangent transform to score candidates:

Here, $\kappa$ is a sharpness parameter (set to 30 by default). The $\tanh$ function saturates at $\pm1$, meaning sentences with strong alignment to the pivot vectors are scored maximally, while weak, boundary-dwelling candidates are heavily suppressed.

3. Diversity-Aware Filtering

Rejection sampling in restricted green regions can lead to repetitive and monotonous generations. To break this robustness-quality trade-off, SAMARK implements a two-stage diversity controller:

• Hard Filtering: Excludes candidates that exceed an $n$-gram overlap threshold ($\theta_{\text{ngram}}$) or a semantic similarity threshold ($\theta_{\text{sem}}$) compared to previously generated sentences.
• Soft Regularization: Adds a Semantic Diversity Bonus $D(x)$ and a Vocabulary Novelty Bonus $N(x)$ to the final sampling score:

The final watermarked sentence is selected via a temperature-scaled softmax distribution over the matched candidate pool.

# Conceptual implementation of SAMARK generation loop for a single sentence
import torch
import torch.nn.functional as F

def samark_sentence_selection(candidates, prev_sentences, pivot_vectors, target_flags, kappa=30.0, lmbda_div=0.35, epsilon=1.0):
    # candidates: list of candidate sentence strings
    # prev_sentences: list of previously generated sentence strings
    # pivot_vectors: Tensor of shape (c, d)
    # target_flags: Tensor of shape (c,) with values in {-1, 1}
    
    valid_candidates = []
    
    # 1. Hard Filtering (Simulated)
    for cand in candidates:
        if is_repetitive_ngram(cand, prev_sentences):
            continue
        if excess_semantic_similarity(cand, prev_sentences):
            continue
        valid_candidates.append(cand)
        
    if not valid_candidates:
        return candidates[0] # Fallback
        
    # Get embeddings for valid candidates
    cand_embeddings = get_embeddings(valid_candidates) # Shape: (N, d)
    
    # 2. Multi-channel Alignment (Cosine Similarities)
    # Cosine similarity matrix shape: (N, c)
    cos_sims = F.cosine_similarity(cand_embeddings.unsqueeze(1), pivot_vectors.unsqueeze(0), dim=-1)
    
    # Check if similarity signs strictly match target flags
    signs = torch.sign(cos_sims)
    matched_mask = (signs == target_flags.unsqueeze(0)).all(dim=-1)
    
    matched_indices = torch.where(matched_mask)[0]
    if len(matched_indices) == 0:
        # Fallback if no candidate strictly matches the multi-channel pattern
        matched_indices = torch.arange(len(valid_candidates))
        
    # 3. Hyperbolic Scoring & Soft Regularization
    scores = []
    for idx in matched_indices:
        # Calculate signed cosine similarity: r_j * cos(E(x), v_j)
        signed_sim = target_flags * cos_sims[idx]
        s_wm = torch.tanh(kappa * signed_sim).sum().item()
        
        # Calculate diversity bonus
        div_bonus = calculate_diversity_bonus(valid_candidates[idx], prev_sentences)
        
        total_score = s_wm + lmbda_div * div_bonus
        scores.append(total_score)
        
    # 4. Softmax Selection
    scores_tensor = torch.tensor(scores)
    probs = F.softmax(scores_tensor / epsilon, dim=-1)
    selected_idx = torch.multinomial(probs, 1).item()
    
    return valid_candidates[matched_indices[selected_idx]]

Key Results

Huo et al. (arXiv 2026) evaluated SAMARK using 500 samples from the BookSum and C4 datasets, using Mistral-Small-3.1-24B-Base-2503 and NVIDIA-Nemotron-Nano-9B-v2-Base as backbones. Paraphrasing attacks (Doc-P I and Doc-P II) were executed using GPT-4.1-mini.

BookSum Dataset Detection Results (TP@FP1%)

Table 2 in the paper reveals the massive robustness gap under aggressive paraphrasing:

As Section 4.2 describes, while the strongest prior semantic baseline (PMark) drops to 57.5% detection rate under Doc-P II, SAMARK maintains a stellar 88.1% TP@FP1% on Mistral-Small. On the Nemotron-Nano model, SAMARK achieves 90.2% under the same attack.

Computational Overhead comparison

The trade-off for semantic-level watermarking is generation latency. Table 4 details the generation and detection overhead:

While SAMARK's generation latency (0.218s per token) is higher than token-level methods due to candidate sentence sampling, it is 1.7x faster than SemStamp and requires fewer sampled tokens. On the detection side, SAMARK is over 130x faster than PMark because it requires only a single forward pass through the embedding model rather than costly LLM re-generation.

Limitations & Open Questions

Despite its breakthrough robustness, SAMARK has several constraints that security engineers must keep in mind before production deployment:

1. Generation Latency: Because LLMs decode token-by-token, SAMARK must generate multiple candidate sentences ($N=64$ in the default budget) before evaluating embeddings and selecting the best match. This adds latency that may be unacceptable for real-time conversational bots.
2. No Formal Distortion-Free Guarantee: The hard-filtering and soft regularization mechanisms slightly alter the output probability distribution of the language model, meaning there is no mathematical guarantee of zero text quality degradation (unlike Gumbel or PRC-based token watermarks).
3. Zero-Bit Limitation: SAMARK is currently designed as a zero-bit detector (it only provides a binary "yes/no" classification) and cannot easily embed multi-bit payloads (such as user IDs) without sacrificing semantic density.

What Practitioners Should Do

If you are deploying text watermarking in a production environment where users are likely to use paraphrasing tools (such as QuillBot or ChatGPT) to evade tracking, follow these implementation guidelines:

1. Transition from Context-Hashed Seeds to Semantic Anchoring: Avoid KGW-style context hashing or SemStamp-style preceding-context hashing. Implement the Self-Anchored Condition where the seed $r_t$ is a deterministic function of the current candidate's sentence embedding.
2. Use High-Quality Sentence Encoders: Deploy a robust, fast sentence embedding model such as all-mpnet-base-v2 or bge-large-en-v1.5 to generate the pivot vectors and verify semantic regions.
3. Configure Hyperbolic Transforms: Set the sharpness parameter $\kappa = 30.0$. This ensures that candidates are pushed away from the decision boundary, preventing sign-flipping when users make minor edits.
4. Balance Diversity and Signal with Soft Regularization: To prevent repetitive phrasing, configure hard thresholds ($\theta_{\text{ngram}} = 0.4$, $\theta_{\text{sem}} = 0.8$) and assign positive weights to soft diversity metrics ($\lambda_{\text{div}} = 0.35$, $\lambda_{\text{nov}} = 0.2$).

The Takeaway

SAMARK proves that the vulnerability of LLM watermarks to paragraph paraphrasing and sentence reordering is not an inherent flaw of semantic watermarking, but rather a consequence of step-dependent seed designs. By anchoring the watermark key to static regions of the semantic space, SAMARK achieves unprecedented robustness without sacrificing generation quality. For organizations building content verification and regulatory compliance tooling, self-anchoring represents the new state-of-the-art paradigm in text provenance.

Den's Take

I've said it before: token-level watermarks are security theater. The moment an adversary runs model outputs through a cheap paraphrasing API, the metadata evaporates. This is why SAMark's shift toward self-anchored semantic spaces is the most practical advancement in content provenance I’ve seen this year. By anchoring keys directly to sentence embeddings rather than relying on a fragile chain of token hashes, it finally addresses the reality of how attackers actually bypass detection.

This isn't just an academic exercise; it has massive commercial implications. Imagine a digital publishing giant facing a $20M copyright infringement dispute because a competitor easily stripped their distribution watermarks using simple paragraph reordering. If we cannot reliably attribute LLM-generated content under basic editing conditions, enterprise compliance and IP protection fall apart.

However, we must look at the broader ecosystem. As I argued in AI Agent Traps: When the Environment Becomes the Attacker, autonomous systems constantly ingest and execute untrusted environmental data. Robust watermarking like SAMark is vital because it provides a reliable, tamper-resistant mechanism to verify whether incoming instructions originated from a trusted enterprise LLM or an external adversary attempting to hijack the agent.