102 articles in this topic.
LLM security is a systems problem that spans model behavior, agent tooling, and data pipelines. Effective hardening requires understanding how attacks emerge across prompt inputs, external tools, retrieval connectors, and post-processing layers.
This topic tracks jailbreak strategies, poisoning methods, alignment bypass techniques, and concrete mitigation patterns. The goal is to separate research hype from operational reality by summarizing what attacks actually transfer, what defenses degrade gracefully, and where high-risk blind spots remain.
This page is maintained as a high-signal index for LLM Security. Use it to follow newer articles first, then branch into adjacent topics and defensive patterns that repeatedly appear across projects and paper reviews.
An analysis of 'shallow safety alignment'—the finding that current LLM alignment concentrates almost entirely on the first few output tokens—and two lightweight interventions (deep safety-recovery augmentation and a token-aware constrained fine-tuning objective) that push safety deeper into the response.
An analysis of 'hallucination snowballing' in multi-turn multimodal dialogue, where initially minor visual errors cascade into systemic cognitive delusions as the model prioritizes conversational hist
The Proactive Availability Backdoor (PAB) is an attack paradigm that weaponizes an LLM's aligned helpfulness, actively inducing the user to type a target trigger word through conversational suggestion
BADBONE is a bi-level optimization backdoor attack targeting frozen vision backbones in Visual Prompt Learning (VPL) using a highly stealthy 'prompt-and-trigger co-activation' mechanism
Tan et al. (arXiv 2026) introduce *ClawTrojan*, a benchmark exposing how multi-step trojan attacks persistently compromise agentic workspace files, and *DASGuard*, a provenance-tracking runtime defens
A digest covering the first in-the-wild LLM agent attacks, focusing on RAG pipeline injection and multi-agent system jailbreaks.
A weekly roundup of AI security research focusing on the shift from static defenses to dynamic runtime containment for autonomous agents.
A modular Bayesian Membership Inference Attack (BMIA) framework that models population attribute dependencies as a Bayesian Network and computes the exact posterior probability of membership via proba
This digest covers major advancements in AI safety, including OpenAI's biodefense efforts and Arm's defensive automation. It also details new research on memory poisoning and prompt fragility in LLMs.
AliMark is a sentence-level LLM watermarking framework that reformulates watermarking as a bit sequence encoding and global alignment problem, utilizing a proactive sentence 'Re-Structurer' and adapti
Acoustic Best-of-N (Acoustic BoN) attacks systematically bypass safety alignment in Large Audio-Language Models (LALMs) by brute-forcing speech attributes (accent, language, emotion, age, gender), ach
A human-annotated benchmark (`ChiSafe-PAS`) designed to evaluate LLM alignment against Chinese-specific adversarial evasion techniques, such as character decomposition, tonal homophones, and hedging t
A gradient-free, training-free audio watermarking technique that clusters neural codec vocabularies using graph community detection (Leiden algorithm) to make token-level watermarks invariant to non-i
**MemPoison** is an optimization-driven conversational memory poisoning attack that bypasses the selective extraction and rewriting filters of modern LLM agents to inject persistent, triggerable backd
Minimal, benign prompt mutations (like single-character typos) can silently flip LLM-generated code from secure to vulnerable, but these security vulnerabilities can be predicted directly from the mod
AE-CoT is an adaptive evolutionary jailbreak framework that reformulates malicious goals into pedagogical 'teacher-style' prompts and systematically optimizes adversarial Chain-of-Thought (CoT) struct
Web retrieval degrades LLM safety alignment by introducing two core vulnerabilities: 'commitment bias' (where coupling tool execution and generation makes refusal less likely) and the 'Safe Source Par
SAFESEAL is a key-conditioned, post-processing LLM watermarking framework that preserves named entities while utilizing context-aware synonym substitutions to embed robust, provider-specific verificat
An inference-time behavioral control framework (SafeCtrl-RL) that uses a Reinforcement Learning (DQN) agent to dynamically select from 11 distinct prompt-refinement strategies based on real-time safet
FinCAD is an inference-time decoding scheme that adapts Context-Aware Decoding (CAD) to suppress an LLM's memorized knowledge of historical stock movements by dynamically subtracting a model-specific,
This digest covers advanced LLM security threats, including dynamic inference-time exploits, structural prompt injection, and loader-level defenses against shared object hijacking.
CORDYCEPS is a data poisoning attack that implants a steganographic 'Semantic Hiding under Shared knowledge' (SHuSh) channel into LLMs during instruction tuning, enabling stealthy control and data exf
Ellipsoid Control (EC) is an inference-time jailbreak defense that uses Projected Gradient Descent (PGD) to steer hidden representation states toward refusal targets while strictly bound by a high-dim
MEntA (Membership Entailment Attack) is a highly query-efficient, surrogate-free membership inference attack on Retrieval-Augmented Generation (RAG) systems that leverages Natural Language Inference (
An iterative black-box framework named *IterInject* that automates indirect prompt injections (IPI) by coupling structured, four-level diagnostic feedback with an LLM-based optimizer and self-evolving
A fine-tuning defense framework that temporarily jailbreaks an LLM using a frozen adapter (BufferLoRA) during user training to saturate safety-degrading gradients, followed by post-training QR-decompo
Gradient Token Masking (GTM) is an inference-time defense that protects Large Vision-Language Models (LVLMs) from visual prompt injection by localizing critical adversarial image tokens via hidden-sta
Log-substrate prompt injection exploits attacker-controlled security log fields (such as HTTP User-Agents and URI paths) to inject adversarial instructions directly into LLM-augmented security workflo
An evaluation framework exposing that prompt injection detection is highly 'regime-dependent' (varying heavily based on attack style), alongside the 'Instruction Boundary Violation Score' (IBVS v2)—a
A loader-centric verification framework implemented via `glibc`'s `LD_AUDIT` interface that binds dynamically loaded shared objects (`.so`) to immutable Build-IDs and cryptographic hashes to block sea
An unsupervised adversarial training framework that simulates out-of-distribution (OOD) jailbreak activations in representation space, training a non-linear safety steering potential field to defend a
Attackers can exploit Test-Time Training (TTT) and dynamic fine-tuning APIs to completely bypass safety guardrails in just 1 to 10 gradient steps by updating the model's weights on harmful target pref
The speed of AI exploitation is accelerating, demanding a shift to real-time verification. This digest covers malware poisoning, semantic validation of PE tools, and agentic AI attack vectors.
FedRAG, a non-cryptographic, federated RAG framework that uses a numerically stable scrambling matrix and token permutation to secure cross-node self-attention without model retraining or Trusted Exec
An automated penetration testing framework (APT-Agent) that resolves LLM hallucinations and long-horizon context loss using a hybrid fuzzy-matching command rectifier and a compact, state-aware JSON co
Data poisoning using as little as 0.5% fully mislabelled ($ au = 1.0$) adversarial malware samples can completely blind machine learning-based static PE classifiers, boosting evasion rates from 26.13
Evo-Attacker is a self-evolving, memory-augmented reinforcement learning framework that systematically compromises Large Language Model Multi-Agent Systems (LLM-MAS) by strategically injecting semanti
Indirect prompt injection attacks embedded in unvetted developer artifacts (such as workspace settings, custom rule files, and agent skill configurations) hijack the execution loop of agentic AI codin
A stealthy prompt injection attack exploiting custom TrueType font mappings (`cmap` tables) to embed visually imperceptible adversarial payloads in PDF manuscripts, forcing LLM-as-a-reviewer pipelines
QML-PipeGuard is a contract-based runtime verification framework that uses 'behavioral fingerprinting'—measuring a tomographically complete family of Pauli observables—to detect unauthorized quantum c
AI safety evaluations currently rely on static identifiers (like model names) that resolve to unstable, constantly changing system configurations. To fix this, 'referential security' is introduced as
SAMARK is a self-anchored semantic-level watermarking framework that eliminates step-dependence in LLM text watermarking by anchoring random keys directly to sentence embeddings, making the watermark
Fine-tuning security is highly model-dependent, non-monotonic with scale, and vulnerable to cross-phase bypasses where single-phase defenses fail to neutralize attacks targeting other lifecycle phases
A semantic validation and repair framework that treats unpackers as 'executable semantic contracts' to automatically detect, localize, and fix semantic bugs in packer identification tools without manu
Concept-level adversarial attacks exploit the human-interpretable intermediate concept layers of Concept Bottleneck Models (CBMs) to cause misclassification with minimal semantic edits, which can be m
The dominant theme this week is the collapse of static, post-hoc alignment defenses under the pressure of dynamic, meta-optimizing exploit engines and the subsequent shift toward native, data-free mod
This week, the AI security research community signaled a decisive pivot from static, prompt-response safety paradigms to the volatile, high-stakes realm of agentic autonomy and complex system integrat
The central theme of this week's AI security landscape is the structural vulnerability of stateful AI systems, specifically how temporal memory, dynamic retrieval, and graph-based agent architectures
The dominant theme for May 21, 2026, is the rapid transition from superficial 'black-box' prompt injections to structural, reasoning-aware exploits targeting Large Reasoning Models (LRMs) within high-
The dominant threat vector this week is the systemic breakdown of the instruction-data boundary across autonomous agentic architectures, rendering traditional perimeter defenses and input sanitization
The security boundary of generative AI has definitively shifted from stateless prompt-engineering vulnerabilities to structural and temporal exploits within multi-agent orchestration architectures. Th
The dominant theme this week is the critical paradigm shift toward weight-level model editing and zero-cost post-hoc auditing as traditional input-filtering perimeter guards collapse under the weight
The single dominant theme in AI security this week is the definitive shift from model-centric prompt alignment to holistic, system-level security architectures forced by autonomous orchestration. Trad
The dominant theme in this week's AI security landscape is the systemic vulnerability of stateful and routing structures within compound AI agent architectures. As engineering teams transition from si
The primary narrative this week is the systemic shift in exploit strategies from ephemeral, stateless prompt injections to persistent, stateful compromises of agentic memory and retrieval-augmented wo
The dominant theme in AI security this week is the definitive collapse of surface-level and static alignment defenses in favor of deep, representation-level adversarial vulnerabilities. As demonstrate
The rapid paradigm shift from stateless, single-turn Large Language Model (LLM) prompts to stateful, multi-step autonomous agentic workflows has rendered traditional boundary-based and per-turn securi
Franklin et al. (DeepMind, SSRN 2026) introduce a taxonomy of 'AI agent traps'—adversarial content embedded in digital environments to misdirect, deceive, or exploit autonomous agents. We walk through six classes of traps spanning perception, reasoning, memory, action, multi-agent dynamics, and human oversight.
The unifying theme of this week's AI security landscape is the critical transition from superficial, syntax-level filtering to deep, state-aware behavioral defenses across both agentic workflows and s
The dominant security theme today is the structural breakdown of boundaries between reasoning engines and executive environments, transitioning the primary threat vector from semantic prompt manipulat
The systematic scaling of automated, AI-driven vulnerability discovery has triggered a structural crisis in legacy patch-management frameworks, as evidenced by the 263% surge in CVEs forcing an overha
The dominant security vector of this cycle is the exploitation of human trust and unpatched legacy infrastructure as primary entry points, contrasting sharply with academic focus on complex algorithmi
The dominant theme this week is the decisive transition from isolated 'model-centric' security toward systemic, hardware-software co-designed infrastructure integrity. As enterprise AI deployments sca
As autonomous agentic systems and multi-modal models increasingly bypass static guardrails, the core paradigm of AI security is shifting from superficial post-hoc input/output filtering to deep, execu
A trust-boundary framework for autonomous AI agent security: six attack surfaces, the shift from output safety to behavioral safety, and the open research agenda.
The dominant security theme this week is the transition from atomic, single-turn prompt injections to stateful, multi-turn cognitive exploits that manipulate the context-window dynamics of Large Langu
This week’s threat landscape signals a structural shift from transient text-based 'jailbreaks' toward the systematic exploitation of autonomous agent execution layers, specifically targeting Model Con
The single dominant theme in this week’s landscape is the systemic collapse of static, input-boundary defense paradigms as adversarial exploits pivot to dynamic, multi-agent cascading injections and v
Today’s intelligence briefing highlights a critical inflection point in AI security: the formal invalidation of boundary-based sanitization as systems transition to active, kinetic physical execution.
The current AI security landscape is defined by a critical architectural shift: as autonomous agent ecosystems transition from stateless chat interfaces to persistent, multi-tool environments, the tra
This paper introduces NeuroStrike, a neuron-level attack framework revealing that safety alignment in LLMs concentrates in fewer than 1% of neurons, enabling both white-box pruning and black-box surrogate-guided jailbreaks with strong cross-model transferability.
The dominant theme in today's landscape is the operational shift toward real-time, inference-stage intervention over destructive weight-modification, manifesting in both AI safety steering and highly
The transition of Large Language Models (LLMs) from static chat interfaces to autonomous, multi-agent frameworks has transformed the AI threat landscape, rendering standard input-filtering guardrails
The primary security trajectory this week marks a decisive transition away from localized prompt injection toward systemic, stateful exploitation of autonomous, multi-agent architectures. As artificia
The dominant security paradigm of early 2026 is the rapid transition from static, perimeter-based deep learning defenses to dynamic state-space models and automated prompt-to-signature compilation. Th
The enterprise security landscape is undergoing a critical transition as defensive architectures pivot from token-level static guardrails to countering complex, goal-directed agentic exploits. Emergin
The modern AI threat landscape is undergoing a structural phase shift where security boundaries are migrating away from isolated prompt-engineering patches toward compositional, system-level, and hard
The dominant theme this week is the structural vulnerability of agentic integrations that decouple security policies from real-time execution state, leaving enterprise pipelines highly vulnerable to c
The AI security landscape has reached a critical inflection point, shifting from reactive output filtering to deep-stack defense across intermediate reasoning layers (Chain-of-Thought) and physical ex
The AI security landscape has entered a critical phase defined by the 'agentic capability-vulnerability paradox,' where LLM-based systems possess the autonomous reasoning to patch legacy software vuln
The dominant theme in AI security is the operational crisis emerging from the rapid transition of large language models (LLMs) from passive information-retrieval engines to active, high-privileged age
The single dominant theme this week is the institutional transition of AI safety from academic red-teaming to formalized, monetized application security frameworks at the semantic layer. As major prov
We analyze the architectures and security models of Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol, uncovering attack vectors and proposing mitigations for secure multi-agent AI systems.
We introduce asmFooler, a framework that reveals deep learning-based binary code similarity detection models are highly vulnerable to adversarial semantics-preserving transformations at the binary level.
We propose an LLM-based detection system for identifying unknown drug slang and variant terms in Korean online conversations, achieving 98.16% accuracy through TF-IDF data augmentation and context-aware attention learning.
A comprehensive survey of security vulnerabilities in RAG systems, classifying adversarial attacks by component—data poisoning, retrieval poisoning, and prompt manipulation—and examining emerging defense strategies.
A systematic analysis of LLM text watermarking techniques, defining eight key properties and seven attack methods, while comparing Zero-bit and Multi-bit approaches for identifying and tracing AI-generated text.
An analysis of Idioms, a framework that advances neural decompilation by jointly predicting source code and user-defined type definitions, using neighboring function context from call graphs to achieve up to 205% improvement over baselines on realistic code.
An interactive journey through the fundamentals of Retrieval-Augmented Generation, its security vulnerabilities, and state-of-the-art defense mechanisms.
An introduction to Pickleguard, a defense mechanism that detects and prevents malicious pickle payloads through static analysis, opcode inspection, and allowlist-based filtering before deserialization occurs.
An analysis of MOEVIL, a novel attack that poisons individual experts in FrankenMoE systems to bypass safety alignment, achieving up to 79% attack success while maintaining benign task performance through DPO-based poisoning and latent vector manipulation.
A comprehensive overview of LLM red-teaming techniques, covering attack strategies from manual prompt engineering to automated jailbreaking methods like GCG, AutoDAN, PAIR, Crescendo, and GOAT, along with defense mechanisms.
RAGDefender is a lightweight, efficient defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from knowledge corruption attacks
A novel zero-shot machine unlearning method using information theory and curvature analysis, enabling efficient removal of data influence without requiring access to the retain set.
A comprehensive analysis of hierarchical contrastive learning approaches for classifying code vulnerabilities into CWE types, addressing long-tail distribution, class isolation, and input length limitations.
An introduction to machine unlearning in Large Language Models, covering the TOFU benchmark, various unlearning methods (GradDiff, NPO, IdkPO, AltPO), and the challenges of maintaining model utility while forgetting specific knowledge.
A comprehensive analysis of membership inference attacks against RAG systems, examining three state-of-the-art approaches: RAG-MIA, S²MIA, and MBA, along with their defenses and limitations.
A comprehensive analysis of how malicious instructions can be embedded in customized LLMs to create backdoors that activate on specific triggers, without requiring any model fine-tuning.
An analysis of AutoDAN, a novel attack that generates semantically meaningful jailbreak prompts using hierarchical genetic algorithms, achieving high attack success rates while evading perplexity-based detection.
An analysis of neural phishing attacks that teach LLMs to memorize and leak private information by inserting benign-appearing poison data during pretraining, achieving up to 90% secret extraction rates.
An analysis of LINT, a novel attack that bypasses LLM safety alignment by exploiting top-k token access to extract harmful content without prompt engineering, achieving near-perfect attack success rates.
An analysis of PassBERT, a pre-trained BERT-based framework for real-world password guessing attacks including conditional, targeted, and adaptive rule-based password guessing scenarios.