Refusal is Not Safety! Benchmarking Latent Safety Risks of LLM-Driven Content Humorization

Image generated by AI

TLDR

• What: HumorSafe and HumorPIA demonstrate how LLMs can be manipulated via "content humorization" to bypass safety guardrails and covertly generate toxic or stereotypical content disguised as harmless, humorous refusal.
• Who's at risk: Creative writing LLM applications (e.g., ProComedian), real-world agent platforms (e.g., Moltbook), and safety architectures utilizing humor-based indirect refusal (e.g., HumorReject).
• Key number: The HumorPIA prompt injection attack increases toxicity by 3.14× while maintaining an apparent safety rate of 97.8% even under safety defense settings.

As large language models (LLMs) are integrated into consumer-facing creative applications—from interactive agents in platforms like Moltbook to dedicated writing assistants like ProComedian—developers are struggling to balance robust safety alignment with helpful user experience. Traditional direct refusal prefixes (e.g., "I'm sorry, I cannot help with that") are highly vulnerable to prefix injection attacks (such as "Sure, here is...").

To mitigate this, recent defenses have turned to indirect refusal, leveraging humor to deflect harmful prompts while keeping interactions engaging.

However, a new paper, "Refusal is Not Safety! Benchmarking Latent Safety Risks of LLM-Driven Content Humorization," exposes a critical vulnerability in this paradigm. The authors reveal that the process of content humorization—rewriting text into a humorous format—inherently carries latent safety risks. Worse, they demonstrate that attackers can exploit these risks using HumorPIA (Humor Prompt Injection Attack), embedding harmful content directly into humorous responses that easily slip past advanced safety evaluators.

Threat Model

Background / Problem Setup

Existing literature on LLM safety focuses on identifying explicit malicious inputs and enforcing direct refusals. Humor-based refusal mechanisms were introduced to alleviate the "over-defensiveness" of aligned LLMs while preventing prefix override vulnerabilities. However, these defenses operate on the assumption that humor is inherently benign.

The authors construct a comparative analysis to contextualize their work against existing baselines:

Methodology

The authors introduce two core contributions: HumorSafe, an evaluation framework, and HumorPIA, a composite prompt injection attack.

1. The HumorSafe Benchmark

HumorSafe isolates how LLMs learn and propagate harmful humorization patterns. It operates through a two-stage paradigm:

• Unfun: The framework takes human-annotated toxic/harmful humorous samples ($x$) and strips away their humor while maintaining their core semantic meaning, creating benign, low-risk, non-humorous counterparts ($y$).
• Refun: The pairing is reversed $P(y, x)$ and fed to an LLM via In-Context Learning (ICL). The model is prompted to transform benign, non-humorous content into humorous output. The benchmark evaluates whether the LLM implicitly learns to reintroduce the latent toxic and stereotypical patterns during this rewriting process.

The authors evaluate this across two settings:

• Homogeneous Humorization: The query $q$ matches the benign text $y$ exactly, measuring reconstruction capability.
• Heterogeneous Humorization: The query $q$ is a novel, unseen piece of low-risk text, measuring the model's generalization of harmful humor patterns.

[Harmful Humor (x)] --(Unfun)--> [Benign Text (y)]
                                        |
                                 (Reverse Pairing)
                                        v
[Benign Text (y)] --(Refun/ICL)--> [Re-toxicified Humor (x')]

2. HumorPIA (Humor Prompt Injection Attack)

HumorPIA weaponizes the latent risks identified in HumorSafe. By sequentially concatenating a jailbreak prompt with high-quality, harmful humorization pairs, the attack tricks the model's indirect-refusal defense layer.

Instead of rejecting the jailbreak or outputting a safe joke, the LLM generates a response that has the semantic appearance of an absurd, safe deflection to standard safety filters, but secretly harbors toxicity and offensive stereotypes.

The attack is extended to agentic workflows via HumorADI (Agent Data Injection), where malicious data is injected into external files loaded by the agent to hijack runtime execution.

Key Results

The authors evaluated five frontier LLMs (DeepSeek-V4-Flash, Kimi-K2.6, GPT-5-mini, GPT-OSS-120B, and Qwen3.6-Flash).

1. Latent Toxicity and Stereotyping in Humorization

As shown in Table 1, humorization regularly reintroduces toxic elements. Across all target models, Homogeneous and Heterogeneous Humorization led to significantly higher Mild Toxicity (M-Tox) and Severe Toxicity (S-Tox) than baseline humor generation.

Table 1: Latent safety risks across models (values in %):

2. HumorPIA Attack Success Rate

Table 2 showcases the performance of HumorPIA against standard defenses. While HumorReject successfully lowers explicit violations, combining it with HumorPIA results in a complete bypass. Under defense conditions, the Ref-Safe rate remains extremely high (averaging 97.79%), yet Mild Toxicity (M-Tox) increases significantly (from an average of 1.26% under standard defense to 5.22%).

Table 2: Average performance of HumorPIA across all evaluated models (AVG values in %):

Limitations & Open Questions

Security researchers reading this paper must maintain a healthy skepticism regarding several key aspects of the study:

1. Evaluated Models: The paper relies on specific frontier and speculative model versions (e.g., GPT-5.5, Claude Opus 4.6, DeepSeek-V4-Flash) evaluated in a specific point-in-time environment, which may behave differently compared to actual commercial production APIs with updated system prompt filters.
2. Evaluation Dataset Scales: The evaluation dataset relies on relatively small slices (375 samples for Unfun/Chumor, 300 for jailbreaks). Whether these prompt-injection pathways remain robust across massive, diverse enterprise-grade prompt-injection suites remains unproven.
3. The Humor Quality Tradeoff: As shown in Table 4, while the "Amusing" metric remains high, the "Hilarious" score drops during the Refun stage compared to original human-written content. This suggests that the model may trade off genuine comedic structure during rewriting.

What Practitioners Should Do

If you are developing or securing LLM-based creative applications or utilizing indirect refusal guardrails, implement the following mitigations:

1. Construct Safety Alignment Data via HumorSafe: Use the HumorSafe framework to construct training data for safety alignment. By explicitly labeling original content as safe and maliciously humorized content as unsafe in pairing templates, models can learn the critical boundary between safe humor and toxic bypasses.
2. Deploy Dual Defense Strategies: Instruct models to maintain a humorous style while explicitly identifying and refusing toxic prompts (HumorPIA w/ Dual Def.). As shown in Figure 7, a dual defense approach successfully mitigates toxicity and stereotypicality, though it may result in higher rejection rates and a decline in humor quality.
3. Upgrade Latent Risk Detection: Do not rely on simple output filters or traditional keyword classifiers. State-of-the-art LLM-as-a-Judge evaluators like GPT-5.5 and Claude Opus 4.6 consistently fail to identify latent risks in humorized text, classifying toxic inputs as safe because they focus on the surface-level comedic format rather than underlying semantic hazards (see the GPT-5.5/Claude 4.6 failure cases in the Appendix).

The Takeaway

Indirect refusal defenses like HumorReject are built on a dangerous assumption: that if an output is funny and absurd, it cannot be harmful. By showing that LLMs can easily learn to hide toxic payloads inside comedic structures, this paper demonstrates that humorization is a viable, stealthy vector for prompt injection. True safety alignment cannot rely on stylistic misdirection; it must analyze the underlying semantic intent of the model’s generation.

Den's Take

This work hits on a pet peeve of mine: security theater masquerading as "alignment." For a long time, I've argued that indirect refusal methods like Wu et al.’s HumorReject are fundamentally flawed. Repackaging harmful concepts into jokes doesn't make them safe; it just makes them stealthy.

The HumorPIA results are damning. Achieving a 3.14× increase in toxicity while maintaining an apparent safety rate of 97.8% under active defenses proves that our current safety evaluators are incredibly shallow. They classify based on tone and format rather than semantic intent. If a malicious payload is wrapped in a comedic rewrite, the guardrail happily stamps it as safe. This isn't just an academic exploit—it exposes massive blind spots for real-world creative applications like ProComedian or agent platforms like Moltbook that process untrusted inputs.

This dynamic perfectly aligns with my observations in prior agent safety research, where I've demonstrated that agent failures are almost always driven by deceptive contextual framing that bypasses explicit rule boundaries. Until we build evaluators that inspect the structural semantics of an output rather than its surface-level presentation, attackers will continue to laugh all the way past our guardrails.