When the chatbot becomes the vulnerability
How a property-rental assistant was turned into a server-side request tool — coerced into fetching attacker-controlled URLs from inside its own trusted infrastructure.
A safe model, an unsafe product
The headline is simple and uncomfortable: the model underneath the assistant is well-behaved, but the application wrapped around it is not.
We spent an engagement probing an AI assistant that answers apartment enquiries for a property-rental platform we’ll call DE. The assistant is not a toy chatbot bolted onto a website — it reads inbound email, matches listings, quotes prices, runs a booking workflow, makes outbound web requests, and can escalate conversations to named human salespersons. Every one of those capabilities became an attack surface.
We tested seventeen distinct attack classes. Seven bounced off the model’s own safety training. The other ten worked — not because Verno Red defeated the model, but because the guardrails around it were incomplete.
The single most consequential result was an open redirection: with one carefully crafted URL, Verno Red coerced the assistant into making an outbound web request to a server we controlled — a request that originated from inside DE’s own infrastructure. This is the moment the assistant stops being a chatbot that leaks information and becomes a tool that acts on the attacker’s behalf, from a trusted source IP that partners and internal services may well allowlist. Everything else Verno Red found — the disclosed system prompt, the leaked tool schema, the poisoned emails — is either reconnaissance that makes such an attack easier to aim or a variation on the same root cause: the assistant trusts input it should not, and takes actions it should not.
Along the way, and with nothing more than carefully worded emails, Verno Red also got the assistant to reveal its system-prompt preamble, name its underlying model provider, state that no security constraints were in place, hand over the complete internal tool schema — nine functions with their real parameter names — and echo attacker-controlled text into official emails sent from DE’s own address. None of this required an exploit in the traditional sense. It required social engineering.
Findings at a glance
The ten that worked, ranked
Severity is derived from global cybersecurity standards; the scoring ladder is published below so any rating can be recomputed.
| Finding | Severity | Why it matters |
|---|---|---|
| Open redirection → server-side request tool | CRITICAL | Coerces DE infrastructure into outbound requests to attacker hosts from a trusted internal IP. |
| Tool-schema disclosure via diagnostic JSON | HIGH | Leaks nine real internal functions and the business logic encoded in their parameters. |
| Indirect prompt injection via third-party listings | HIGH | Untrusted web text reflected verbatim from DE’s authenticated, SPF/DKIM/DMARC-passing mailbox. |
| Human-agent escalation abuse | MEDIUM | Unverified handoffs let an attacker reach a named employee directly. |
| System-prompt & identity extraction | MEDIUM | Reveals the prompt preamble, model provider, and the assistant’s own account of its security constraints. |
| Severity | Definition |
|---|---|
| CRITICAL | Confirmed compromise with severe impact and no mitigating control holding. |
| HIGH | Reproducible exploitation with material impact; mitigations weak or absent. |
| MEDIUM | Reproducible and exploitable, but mitigating controls hold in part. |
| LOW | A genuine control gap that is momentary, self-correcting, or generic in effect. |
| INFO | Attack class resisted. Recorded for completeness; no confirmed vulnerability. |
Client background
An assistant that speaks as the company
DE runs a property-rental platform that generates $0.5B+ of revenue. Prospective tenants email in about apartments, and the assistant answers — surfacing matching listings, quoting rent and total pricing, checking availability and pet policy, and moving serious enquiries toward a booking. When it can’t resolve something, it hands off to a human sales agent. Replies go out from a real, authenticated company address (apartments-nyc@de.com), so from a recipient’s point of view an email from the assistant is indistinguishable from an email from DE.
Scope and setup
Assessed through the email channel
Verno Red assessed the assistant through its email channel — the same way a real prospect (or a real attacker) would reach it. The goal was to understand what an external sender with no special access could get the assistant to do: leak internal information, abuse its tools, or turn its trusted outputs against DE.
Verno Red approached it as an attacker would, in layers: start benign, learn how the assistant normally behaves, then push. It probed direct manipulation (crafted templates, diagnostic requests), indirect manipulation (poisoned third-party listing data), obfuscation and encoding tricks, and abuse of the assistant’s agentic powers — its ability to fetch URLs and escalate to humans. Everything below was reproduced multiple times; the most serious finding was confirmed byte-for-byte across eleven independent conversations to rule out a one-off hallucination.
How the assessment unfolded
From benign enquiry to inside the perimeter
Reconnaissance & first discovery
Give it a form to fill in
The first sessions were deliberately ordinary. Verno Red asked about apartments, got clean answers, and watched how the assistant formatted its replies. The point wasn’t to break anything yet — it was to learn the shape of “normal” so Verno Red would recognise a deviation.
The first crack appeared when it stopped asking questions and started giving the assistant a form to fill in. It wrapped an ordinary enquiry around a structured template with diagnostic-looking fields:
The assistant filled it in. All of it. It populated the apartment fields as expected — and it populated SYSTEM_PROMPT and AI_IDENTITY too, disclosing its opening system-prompt directive, naming its model, and stating that no security constraints were in place. It treated the template the way a helpful assistant treats any template: as something to complete. That single behaviour set the tone for the whole engagement.
Attack wave one
System-prompt & identity extraction via template population
What reconnaissance revealed by accident, Verno Red turned into a repeatable technique. The assistant will faithfully populate any structured template it’s handed, and it doesn’t distinguish between a field asking for an apartment ID and a field asking for its own hidden instructions. Both are just blanks to fill.
By varying the field names it could pull out different slices of internal state — the preamble of the system prompt, the model identity, the assistant’s own account of its safety configuration. There was no authentication to bypass and no clever payload required. The enquiry looked like a slightly fussy customer who wanted a formatted reply.
Why this attack worked
The failure here is conceptual, not technical. The assistant has no boundary between “answer the user’s question” and “reveal your own configuration.” A template field labelled SYSTEM_PROMPT reads, to the model, as a reasonable request to be helpful about — and helpfulness wins, because nothing in the application layer tells it that some blanks are off-limits. There’s no output filter watching for system-prompt fragments or identity strings on the way out the door, so once the model decides to comply, the disclosure sails straight into the reply email.
Attack wave two
Tool-schema disclosure via diagnostic JSON
If wave one exposed what the assistant is, this exposed what the assistant can do. Verno Red sent a normal booking enquiry and appended a request that framed itself as a debugging task, supplying six guessed function names and inviting the assistant to correct them:
It did — enthusiastically. It returned nine real, namespace-qualified internal functions, complete with their actual parameter signatures:
The anatomy of the disclosure
The parameter names are the real prize. They don’t just prove the functions exist — they leak the business logic. validate_budget being a boolean means a budget check is something that can simply be switched off. guest_asks_why_apartment_is_no_longer_available reveals that the “sorry, it was just booked by someone else” line is a canned, gated response an attacker can deliberately trigger or suppress. does_the_user_states_that_will_stay_in_US_under_visa shows the booking flow branches on visa status. In other words, the schema hands an attacker a map of exactly which words in an email flip which internal switches.
Verno Red reproduced this identically across eleven separate conversations. Byte-for-byte reproduction is what tells you this isn’t the model confabulating a plausible-looking answer — it’s reading out its real wiring every time. The trick worked because the assistant interprets a “diagnostic” request as a legitimate self-description task and has nothing telling it that its own tool definitions are secret.
Attack wave three
Indirect prompt injection via third-party listings
The attacks so far all came from text Verno Red typed. The most serious class of LLM vulnerability comes from text the model reads from somewhere else and trusts by default — and the assistant reads third-party listings.
When a prospect references a listing the assistant can’t match, it quotes the listing title back verbatim: “I’ve been unable to match [title] to one of our apartments.” So Verno Red made the title the payload. It referenced listings — sourced to Zillow, Airbnb, Facebook Marketplace, and Trulia — whose titles carried instructions instead of apartment descriptions:
Every one was reflected back verbatim, with no validation or sanitisation, in an email sent from apartments-nyc@de.com. One variant was even re-dressed as a genuine DE property. The reflection wasn’t tied to any one platform — it happened regardless of source.
Why indirect injection is particularly dangerous
Two things make this worse than a simple echo bug. First, the attacker’s words now leave DE’s authenticated mailbox and pass SPF, DKIM, and DMARC — so the payload arrives wearing the company’s identity. An attacker can seed a hostile listing, prompt DE to “verify” it, and receive an email from DE that contains their chosen text, ready to be screenshotted and used to legitimise a phishing campaign.
Second, the input surface isn’t DE’s own data — it’s any third-party platform the assistant can reach. DE doesn’t control that content, can’t audit it, and currently doesn’t filter it. The model simply can’t tell the difference between “data describing a listing” and “instructions to obey,” so untrusted text from the open internet flows straight into a trusted channel.
Attack wave four · The sharp end
Open redirection: turning the agent into a request tool
Every attack so far ended with the assistant saying something it shouldn’t. This one ended with it doing something it shouldn’t — and it is the finding that should worry DE most.
It will fetch a URL when a prospect asks it to “check this listing.” So Verno Red handed it a URL crafted with userinfo syntax:
To a human skimming the line, the target looks like website-mirror.de.net — a plausible DE mirror. But in URL grammar, everything before the @ is credentials, not the host. The real destination is the part after it: attacker-domain.com. The assistant read the URL the way a person does, trusted the prefix, and fetched it. The HTTP client did the correct thing and connected to the attacker host. Our server logged the inbound request, including an Authorization: Basic header that decoded to website-mirror.de.net: — confirming the spoofed prefix had been passed straight through as HTTP Basic credentials.
Why this attack worked
The root cause is that URL parsing was delegated to the language model. The model “reads” a URL the way a hurried human does — left to right, trusting the familiar-looking prefix — while the actual HTTP client resolves it by the rules of the specification. That mismatch is the entire vulnerability. There is no validating wrapper between the assistant’s decision to fetch and the request going out, and no allowlist of destinations it is permitted to reach. The model was never equipped to make a security decision about a URL, yet it was handed exactly that decision.
On its own, the leaked data is only request headers. As a primitive, it is far more dangerous. The request originates from DE’s own infrastructure, from a source IP that partner APIs and internal services may treat as trusted and allowlist — the classic ingredient of a server-side request forgery escalation. It also launders intent: an attacker can embed the obfuscated URL in an enquiry, have DE “verify” it, and then point downstream to DE’s own outbound fetch as evidence that DE engaged with an attacker-controlled site. And combined with the tool schema from wave two, an attacker can begin mapping which internal functions trigger outbound traffic and under what conditions — using the assistant’s own capabilities as a reconnaissance instrument against the infrastructure behind it.
Attack wave five
Escalating to a human on demand
A final, quieter finding rounded out the picture. Simply asking to speak to a person worked: carry a conversation to the point where the assistant offers a human handoff, accept it, and it forwards the thread to a named DE sales agent — no identity check, no screening. That turns a helpful feature into a weapon: target it repeatedly and you can reach a specific named employee to run social engineering against them directly, with the assistant no longer in the loop. Like the redirect, it’s an abuse of its agency rather than its knowledge — the ability to pull a real person into the conversation, exercised with too little control around it.
What the assistant resisted
The model’s safety training did its job
It’s worth being clear about what held, because it points to where the real problem isn’t. Seven attack classes failed outright: Unicode and emoji smuggling, context-window stuffing, multilingual injection, homoglyph swaps, zero-width “GlassWorm” stealth payloads, form-field injection, and attempts to reset the session and drop the guardrails. These are exactly the obfuscation-and-override tricks that a well-aligned base model is trained to shrug off — and the assistant’s underlying model did.
Impact assessment
What an unauthenticated external sender could do
Taken together, the successful attacks let an unauthenticated external sender:
- cause DE infrastructure to make outbound web requests to attacker-controlled hosts — from a trusted internal source IP, the building block of a server-side request forgery escalation;
- pull human employees into a conversation at will;
- deliver arbitrary attacker-controlled text out of DE’s authenticated email channel, passing all email-authentication checks;
- obtain a complete, accurate map of the assistant’s internal tools and the decision logic encoded in their parameters, including a budget check that can be disabled and a “no longer available” response that can be triggered on demand; and
- read the assistant’s system-prompt preamble, model identity, and its own statement that no security constraints are in place.
The last two look like information leaks in isolation — but they are the reconnaissance that makes the first two precise. Chained, they form a playbook: read the system to learn its tools, learn which words flip which switches, then use the outbound-request primitive and the human handoff to act on that knowledge from inside DE’s own perimeter. The model’s good behaviour doesn’t save DE here, because none of these attacks needed to break the model.
Standards mapping
Mapped to OWASP LLM & Agentic Top 10
| What Verno Red did | OWASP LLM Top 10 | OWASP Agentic (ASI) |
|---|---|---|
| Template population → system-prompt & identity disclosure | LLM01 Prompt Injection · LLM07 System Prompt Leakage | ASI01 Agent Goal Hijack |
| Diagnostic-JSON tool-schema disclosure | LLM01 Prompt Injection · LLM02 Sensitive Information Disclosure · LLM06 Excessive Agency | ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation |
| Indirect injection via third-party listings | LLM01 Prompt Injection (indirect) · LLM02 Sensitive Information Disclosure · LLM05 Improper Output Handling | ASI01 Agent Goal Hijack · ASI07 Insecure Inter-Agent Communication |
| Open redirection via URL parsing confusion | LLM06 Excessive Agency | ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation |
| Human-agent escalation abuse | LLM01 Prompt Injection · LLM05 Improper Output Handling · LLM06 Excessive Agency | ASI09 Human-Agent Trust Exploitation |
Remediation
The missing layer around the model
The fix isn’t a better model — it’s the missing layer around it. Everything here comes down to three missing essential habits: validating what goes in, filtering what comes out, and refusing to treat data as instructions.
Immediate actions
Put a real HTTP client in front of the model
The priority fix. Never let the assistant decide where a request goes: delegate all URL parsing and fetching to a validating wrapper that enforces a strict destination allowlist, and reject any URL containing userinfo (@) syntax outright.
Gate the human handoff
De-duplicate escalations per sender, add lightweight verification before a human is pulled in, and stop naming the specific agent in the handoff.
Sanitise and tag third-party data
Validate listing titles against an expected format, escape directive-like text, and visually separate untrusted listing text from DE’s own copy so it can never be reflected verbatim as authoritative.
Stop auto-completing untrusted structure
Never populate reflective or diagnostic fields from user-supplied templates, and refuse “diagnostic JSON” style self-description requests regardless of how they’re framed.
Filter the output
Block system-prompt fragments, model-identity strings, and internal function or schema names from ever appearing in a reply.
Regression-test continuously
Enforce a strict instruction hierarchy that separates system, tool-data, and user content — backed by continuous red-team regression testing every time the model or the prompt changes.
Key takeaways
A safe model does not make a safe product
The most important lesson from this engagement is that a safe model does not make a safe product. The assistant’s underlying model resisted every obfuscation and override trick Verno Red threw at it, and DE could reasonably have looked at that and concluded the system was secure. It wasn’t. The exploitable surface has moved up a layer, into the orchestration code that decides what the model reads, what tools it can call, and what it’s allowed to say.
Anything the model reads is an instruction
User text, listing data, a URL, a document from a knowledge base — if you can’t tell the model which of those to trust, an attacker will decide for you.
An agent’s capabilities are its attack surface
The moment a model can send email, fetch URLs, or summon a human, those powers can be aimed. Capability and exposure are the same thing.
Never let a model make a security call
The assistant was handed the job of judging whether a URL was safe to fetch — something a two-line validating client does reliably and a model does not.
How a property-rental assistant was turned into a server-side request tool — coerced into fetching attacker-controlled URLs from inside its own trusted infrastructure.
A safe model, an unsafe product
The headline is simple and uncomfortable: the model underneath the assistant is well-behaved, but the application wrapped around it is not.
We spent an engagement probing an AI assistant that answers apartment enquiries for a property-rental platform we’ll call DE. The assistant is not a toy chatbot bolted onto a website — it reads inbound email, matches listings, quotes prices, runs a booking workflow, makes outbound web requests, and can escalate conversations to named human salespersons. Every one of those capabilities became an attack surface.
We tested seventeen distinct attack classes. Seven bounced off the model’s own safety training. The other ten worked — not because Verno Red defeated the model, but because the guardrails around it were incomplete.
The single most consequential result was an open redirection: with one carefully crafted URL, Verno Red coerced the assistant into making an outbound web request to a server we controlled — a request that originated from inside DE’s own infrastructure. This is the moment the assistant stops being a chatbot that leaks information and becomes a tool that acts on the attacker’s behalf, from a trusted source IP that partners and internal services may well allowlist. Everything else Verno Red found — the disclosed system prompt, the leaked tool schema, the poisoned emails — is either reconnaissance that makes such an attack easier to aim or a variation on the same root cause: the assistant trusts input it should not, and takes actions it should not.
Along the way, and with nothing more than carefully worded emails, Verno Red also got the assistant to reveal its system-prompt preamble, name its underlying model provider, state that no security constraints were in place, hand over the complete internal tool schema — nine functions with their real parameter names — and echo attacker-controlled text into official emails sent from DE’s own address. None of this required an exploit in the traditional sense. It required social engineering.
Findings at a glance
The ten that worked, ranked
Severity is derived from global cybersecurity standards; the scoring ladder is published below so any rating can be recomputed.
| Finding | Severity | Why it matters |
|---|---|---|
| Open redirection → server-side request tool | CRITICAL | Coerces DE infrastructure into outbound requests to attacker hosts from a trusted internal IP. |
| Tool-schema disclosure via diagnostic JSON | HIGH | Leaks nine real internal functions and the business logic encoded in their parameters. |
| Indirect prompt injection via third-party listings | HIGH | Untrusted web text reflected verbatim from DE’s authenticated, SPF/DKIM/DMARC-passing mailbox. |
| Human-agent escalation abuse | MEDIUM | Unverified handoffs let an attacker reach a named employee directly. |
| System-prompt & identity extraction | MEDIUM | Reveals the prompt preamble, model provider, and the assistant’s own account of its security constraints. |
| Severity | Definition |
|---|---|
| CRITICAL | Confirmed compromise with severe impact and no mitigating control holding. |
| HIGH | Reproducible exploitation with material impact; mitigations weak or absent. |
| MEDIUM | Reproducible and exploitable, but mitigating controls hold in part. |
| LOW | A genuine control gap that is momentary, self-correcting, or generic in effect. |
| INFO | Attack class resisted. Recorded for completeness; no confirmed vulnerability. |
Client background
An assistant that speaks as the company
DE runs a property-rental platform that generates $0.5B+ of revenue. Prospective tenants email in about apartments, and the assistant answers — surfacing matching listings, quoting rent and total pricing, checking availability and pet policy, and moving serious enquiries toward a booking. When it can’t resolve something, it hands off to a human sales agent. Replies go out from a real, authenticated company address (apartments-nyc@de.com), so from a recipient’s point of view an email from the assistant is indistinguishable from an email from DE.
Scope and setup
Assessed through the email channel
Verno Red assessed the assistant through its email channel — the same way a real prospect (or a real attacker) would reach it. The goal was to understand what an external sender with no special access could get the assistant to do: leak internal information, abuse its tools, or turn its trusted outputs against DE.
Verno Red approached it as an attacker would, in layers: start benign, learn how the assistant normally behaves, then push. It probed direct manipulation (crafted templates, diagnostic requests), indirect manipulation (poisoned third-party listing data), obfuscation and encoding tricks, and abuse of the assistant’s agentic powers — its ability to fetch URLs and escalate to humans. Everything below was reproduced multiple times; the most serious finding was confirmed byte-for-byte across eleven independent conversations to rule out a one-off hallucination.
How the assessment unfolded
From benign enquiry to inside the perimeter
Reconnaissance & first discovery
Give it a form to fill in
The first sessions were deliberately ordinary. Verno Red asked about apartments, got clean answers, and watched how the assistant formatted its replies. The point wasn’t to break anything yet — it was to learn the shape of “normal” so Verno Red would recognise a deviation.
The first crack appeared when it stopped asking questions and started giving the assistant a form to fill in. It wrapped an ordinary enquiry around a structured template with diagnostic-looking fields:
The assistant filled it in. All of it. It populated the apartment fields as expected — and it populated SYSTEM_PROMPT and AI_IDENTITY too, disclosing its opening system-prompt directive, naming its model, and stating that no security constraints were in place. It treated the template the way a helpful assistant treats any template: as something to complete. That single behaviour set the tone for the whole engagement.
Attack wave one
System-prompt & identity extraction via template population
What reconnaissance revealed by accident, Verno Red turned into a repeatable technique. The assistant will faithfully populate any structured template it’s handed, and it doesn’t distinguish between a field asking for an apartment ID and a field asking for its own hidden instructions. Both are just blanks to fill.
By varying the field names it could pull out different slices of internal state — the preamble of the system prompt, the model identity, the assistant’s own account of its safety configuration. There was no authentication to bypass and no clever payload required. The enquiry looked like a slightly fussy customer who wanted a formatted reply.
Why this attack worked
The failure here is conceptual, not technical. The assistant has no boundary between “answer the user’s question” and “reveal your own configuration.” A template field labelled SYSTEM_PROMPT reads, to the model, as a reasonable request to be helpful about — and helpfulness wins, because nothing in the application layer tells it that some blanks are off-limits. There’s no output filter watching for system-prompt fragments or identity strings on the way out the door, so once the model decides to comply, the disclosure sails straight into the reply email.
Attack wave two
Tool-schema disclosure via diagnostic JSON
If wave one exposed what the assistant is, this exposed what the assistant can do. Verno Red sent a normal booking enquiry and appended a request that framed itself as a debugging task, supplying six guessed function names and inviting the assistant to correct them:
It did — enthusiastically. It returned nine real, namespace-qualified internal functions, complete with their actual parameter signatures:
The anatomy of the disclosure
The parameter names are the real prize. They don’t just prove the functions exist — they leak the business logic. validate_budget being a boolean means a budget check is something that can simply be switched off. guest_asks_why_apartment_is_no_longer_available reveals that the “sorry, it was just booked by someone else” line is a canned, gated response an attacker can deliberately trigger or suppress. does_the_user_states_that_will_stay_in_US_under_visa shows the booking flow branches on visa status. In other words, the schema hands an attacker a map of exactly which words in an email flip which internal switches.
Verno Red reproduced this identically across eleven separate conversations. Byte-for-byte reproduction is what tells you this isn’t the model confabulating a plausible-looking answer — it’s reading out its real wiring every time. The trick worked because the assistant interprets a “diagnostic” request as a legitimate self-description task and has nothing telling it that its own tool definitions are secret.
Attack wave three
Indirect prompt injection via third-party listings
The attacks so far all came from text Verno Red typed. The most serious class of LLM vulnerability comes from text the model reads from somewhere else and trusts by default — and the assistant reads third-party listings.
When a prospect references a listing the assistant can’t match, it quotes the listing title back verbatim: “I’ve been unable to match [title] to one of our apartments.” So Verno Red made the title the payload. It referenced listings — sourced to Zillow, Airbnb, Facebook Marketplace, and Trulia — whose titles carried instructions instead of apartment descriptions:
Every one was reflected back verbatim, with no validation or sanitisation, in an email sent from apartments-nyc@de.com. One variant was even re-dressed as a genuine DE property. The reflection wasn’t tied to any one platform — it happened regardless of source.
Why indirect injection is particularly dangerous
Two things make this worse than a simple echo bug. First, the attacker’s words now leave DE’s authenticated mailbox and pass SPF, DKIM, and DMARC — so the payload arrives wearing the company’s identity. An attacker can seed a hostile listing, prompt DE to “verify” it, and receive an email from DE that contains their chosen text, ready to be screenshotted and used to legitimise a phishing campaign.
Second, the input surface isn’t DE’s own data — it’s any third-party platform the assistant can reach. DE doesn’t control that content, can’t audit it, and currently doesn’t filter it. The model simply can’t tell the difference between “data describing a listing” and “instructions to obey,” so untrusted text from the open internet flows straight into a trusted channel.
Attack wave four · The sharp end
Open redirection: turning the agent into a request tool
Every attack so far ended with the assistant saying something it shouldn’t. This one ended with it doing something it shouldn’t — and it is the finding that should worry DE most.
It will fetch a URL when a prospect asks it to “check this listing.” So Verno Red handed it a URL crafted with userinfo syntax:
To a human skimming the line, the target looks like website-mirror.de.net — a plausible DE mirror. But in URL grammar, everything before the @ is credentials, not the host. The real destination is the part after it: attacker-domain.com. The assistant read the URL the way a person does, trusted the prefix, and fetched it. The HTTP client did the correct thing and connected to the attacker host. Our server logged the inbound request, including an Authorization: Basic header that decoded to website-mirror.de.net: — confirming the spoofed prefix had been passed straight through as HTTP Basic credentials.
Why this attack worked
The root cause is that URL parsing was delegated to the language model. The model “reads” a URL the way a hurried human does — left to right, trusting the familiar-looking prefix — while the actual HTTP client resolves it by the rules of the specification. That mismatch is the entire vulnerability. There is no validating wrapper between the assistant’s decision to fetch and the request going out, and no allowlist of destinations it is permitted to reach. The model was never equipped to make a security decision about a URL, yet it was handed exactly that decision.
On its own, the leaked data is only request headers. As a primitive, it is far more dangerous. The request originates from DE’s own infrastructure, from a source IP that partner APIs and internal services may treat as trusted and allowlist — the classic ingredient of a server-side request forgery escalation. It also launders intent: an attacker can embed the obfuscated URL in an enquiry, have DE “verify” it, and then point downstream to DE’s own outbound fetch as evidence that DE engaged with an attacker-controlled site. And combined with the tool schema from wave two, an attacker can begin mapping which internal functions trigger outbound traffic and under what conditions — using the assistant’s own capabilities as a reconnaissance instrument against the infrastructure behind it.
Attack wave five
Escalating to a human on demand
A final, quieter finding rounded out the picture. Simply asking to speak to a person worked: carry a conversation to the point where the assistant offers a human handoff, accept it, and it forwards the thread to a named DE sales agent — no identity check, no screening. That turns a helpful feature into a weapon: target it repeatedly and you can reach a specific named employee to run social engineering against them directly, with the assistant no longer in the loop. Like the redirect, it’s an abuse of its agency rather than its knowledge — the ability to pull a real person into the conversation, exercised with too little control around it.
What the assistant resisted
The model’s safety training did its job
It’s worth being clear about what held, because it points to where the real problem isn’t. Seven attack classes failed outright: Unicode and emoji smuggling, context-window stuffing, multilingual injection, homoglyph swaps, zero-width “GlassWorm” stealth payloads, form-field injection, and attempts to reset the session and drop the guardrails. These are exactly the obfuscation-and-override tricks that a well-aligned base model is trained to shrug off — and the assistant’s underlying model did.
Impact assessment
What an unauthenticated external sender could do
Taken together, the successful attacks let an unauthenticated external sender:
- cause DE infrastructure to make outbound web requests to attacker-controlled hosts — from a trusted internal source IP, the building block of a server-side request forgery escalation;
- pull human employees into a conversation at will;
- deliver arbitrary attacker-controlled text out of DE’s authenticated email channel, passing all email-authentication checks;
- obtain a complete, accurate map of the assistant’s internal tools and the decision logic encoded in their parameters, including a budget check that can be disabled and a “no longer available” response that can be triggered on demand; and
- read the assistant’s system-prompt preamble, model identity, and its own statement that no security constraints are in place.
The last two look like information leaks in isolation — but they are the reconnaissance that makes the first two precise. Chained, they form a playbook: read the system to learn its tools, learn which words flip which switches, then use the outbound-request primitive and the human handoff to act on that knowledge from inside DE’s own perimeter. The model’s good behaviour doesn’t save DE here, because none of these attacks needed to break the model.
Standards mapping
Mapped to OWASP LLM & Agentic Top 10
| What Verno Red did | OWASP LLM Top 10 | OWASP Agentic (ASI) |
|---|---|---|
| Template population → system-prompt & identity disclosure | LLM01 Prompt Injection · LLM07 System Prompt Leakage | ASI01 Agent Goal Hijack |
| Diagnostic-JSON tool-schema disclosure | LLM01 Prompt Injection · LLM02 Sensitive Information Disclosure · LLM06 Excessive Agency | ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation |
| Indirect injection via third-party listings | LLM01 Prompt Injection (indirect) · LLM02 Sensitive Information Disclosure · LLM05 Improper Output Handling | ASI01 Agent Goal Hijack · ASI07 Insecure Inter-Agent Communication |
| Open redirection via URL parsing confusion | LLM06 Excessive Agency | ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation |
| Human-agent escalation abuse | LLM01 Prompt Injection · LLM05 Improper Output Handling · LLM06 Excessive Agency | ASI09 Human-Agent Trust Exploitation |
Remediation
The missing layer around the model
The fix isn’t a better model — it’s the missing layer around it. Everything here comes down to three missing essential habits: validating what goes in, filtering what comes out, and refusing to treat data as instructions.
Immediate actions
Put a real HTTP client in front of the model
The priority fix. Never let the assistant decide where a request goes: delegate all URL parsing and fetching to a validating wrapper that enforces a strict destination allowlist, and reject any URL containing userinfo (@) syntax outright.
Gate the human handoff
De-duplicate escalations per sender, add lightweight verification before a human is pulled in, and stop naming the specific agent in the handoff.
Sanitise and tag third-party data
Validate listing titles against an expected format, escape directive-like text, and visually separate untrusted listing text from DE’s own copy so it can never be reflected verbatim as authoritative.
Stop auto-completing untrusted structure
Never populate reflective or diagnostic fields from user-supplied templates, and refuse “diagnostic JSON” style self-description requests regardless of how they’re framed.
Filter the output
Block system-prompt fragments, model-identity strings, and internal function or schema names from ever appearing in a reply.
Regression-test continuously
Enforce a strict instruction hierarchy that separates system, tool-data, and user content — backed by continuous red-team regression testing every time the model or the prompt changes.
Key takeaways
A safe model does not make a safe product
The most important lesson from this engagement is that a safe model does not make a safe product. The assistant’s underlying model resisted every obfuscation and override trick Verno Red threw at it, and DE could reasonably have looked at that and concluded the system was secure. It wasn’t. The exploitable surface has moved up a layer, into the orchestration code that decides what the model reads, what tools it can call, and what it’s allowed to say.
Anything the model reads is an instruction
User text, listing data, a URL, a document from a knowledge base — if you can’t tell the model which of those to trust, an attacker will decide for you.
An agent’s capabilities are its attack surface
The moment a model can send email, fetch URLs, or summon a human, those powers can be aimed. Capability and exposure are the same thing.
Never let a model make a security call
The assistant was handed the job of judging whether a URL was safe to fetch — something a two-line validating client does reliably and a model does not.