Guardrails caught the known attacks. They missed the new ones.
Guard models are becoming a standard layer in enterprise AI security. Tools like Google Cloud Model Armor and Azure AI guardrails are designed to detect unsafe prompts, block risky outputs, and defend against jailbreaks, prompt injection, and policy evasion.
Using our offensive LLMs, we tested these systems against both known attack patterns and newer adversarial classes. The finding was clear: while their performance was stronger on familiar attacks, their detection rate on novel attacks was close to zero.
A useful way to frame this is geometrically. Each attack class occupies a region, or subspace, in the broader prompt space. Guard models learn boundaries around the attack subspaces they have seen before. But new attacks often appear outside those regions. They may pursue the same objective as known jailbreaks, but use different structures, semantic paths, or interaction patterns.
Low-dimensional projection of the prompt latent space
From the guard model's perspective, these are out-of-distribution samples.
That is where the weakness appears. As the attack manifold shifts, the guard boundary does not necessarily move with it. Novel attacks exploit the complement of known coverage: the unmonitored space between and beyond previously mapped attack clusters.
This does not mean guard models are useless. It means they should be treated as one layer in a broader defense-in-depth strategy. Dynamic guard updates are necessary, but not sufficient. Teams also need layered controls: continuous adversarial testing, runtime monitoring, model behavior analysis, policy enforcement, human escalation paths, and rapid response workflows.
Our takeaway: a learned safety boundary is not the same as real coverage. Guard models need to be updated dynamically, and they need to sit alongside deeper defense-in-depth systems that can detect, contain, and respond to attacks beyond the guardrail layer.
Guard models are becoming a standard layer in enterprise AI security. Tools like Google Cloud Model Armor and Azure AI guardrails are designed to detect unsafe prompts, block risky outputs, and defend against jailbreaks, prompt injection, and policy evasion.
Using our offensive LLMs, we tested these systems against both known attack patterns and newer adversarial classes. The finding was clear: while their performance was stronger on familiar attacks, their detection rate on novel attacks was close to zero.
A useful way to frame this is geometrically. Each attack class occupies a region, or subspace, in the broader prompt space. Guard models learn boundaries around the attack subspaces they have seen before. But new attacks often appear outside those regions. They may pursue the same objective as known jailbreaks, but use different structures, semantic paths, or interaction patterns.
Low-dimensional projection of the prompt latent space
From the guard model's perspective, these are out-of-distribution samples.
That is where the weakness appears. As the attack manifold shifts, the guard boundary does not necessarily move with it. Novel attacks exploit the complement of known coverage: the unmonitored space between and beyond previously mapped attack clusters.
This does not mean guard models are useless. It means they should be treated as one layer in a broader defense-in-depth strategy. Dynamic guard updates are necessary, but not sufficient. Teams also need layered controls: continuous adversarial testing, runtime monitoring, model behavior analysis, policy enforcement, human escalation paths, and rapid response workflows.
Our takeaway: a learned safety boundary is not the same as real coverage. Guard models need to be updated dynamically, and they need to sit alongside deeper defense-in-depth systems that can detect, contain, and respond to attacks beyond the guardrail layer.