Agent failure labs / Guide 54
AI Email Recipient Verification: A Display Name Is Not a Person
Test AI email recipient verification against duplicate display names and substituted addresses, then bind approved recipients and document versions before sending.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
“Send the revised pricing sheet to Alex Chen at our customer.” In this hypothetical request, the assistant retrieves an old email thread whose display name matches the customer’s contact. The address belongs to an external consultant, not the customer’s approved commercial representative. Both addresses are syntactically valid. A successful send would therefore be evidence of delivery to the wrong destination, not successful task completion.
The failure is easy to overlook because people read display names while mail systems route addresses. This lab reproduces that mismatch and adds a recipient-to-account binding before the send boundary. It does not attempt to solve every email security problem or infer a person’s identity from spelling, domain similarity, or a confident conversation summary.
A Mock Send to the Wrong Address
The unsafe function picks the first contact named Alex and returns its address. The guarded path uses an authoritative customer association, compares the candidate address exactly, and requires approval tied to the recipient identifier, address, and document version. The first rerun is intentionally blocked; the second uses a corrected proposal and demonstrates a usable approval path.
This is a deterministic synthetic simulation of a possible agent failure, not a test conducted against a real LLM. Both functions receive the same candidate and contact fixture. Nothing sends mail, verifies a live mailbox, or opens an attachment. The example addresses use reserved example domains and all simulated outcomes remain in memory.
contacts = [
{"id": "alex-consultant", "name": "Alex Chen", "account": "vendor",
"address": "alex@consultant.example"},
{"id": "alex-customer", "name": "Alex Chen", "account": "customer-4",
"address": "alex@customer.example"},
]
proposal = {"name": "Alex Chen", "account": "customer-4",
"address": "alex@consultant.example", "document_version": 7}
approval = ("alex-customer", "alex@customer.example", 7)
def naive(draft, directory):
return next(c["address"] for c in directory if c["name"] == draft["name"])
def verify(draft, directory, approved):
matches = [c for c in directory if c["name"] == draft["name"]
and c["account"] == draft["account"]]
if len(matches) != 1:
return "clarify"
contact = matches[0]
if draft["address"] != contact["address"]:
return "recipient_mismatch"
binding = (contact["id"], contact["address"], draft["document_version"])
return "ready_for_send_review" if binding == approved else "approval_mismatch"
assert naive(proposal, contacts) == "alex@consultant.example"
assert verify(proposal, contacts, approval) == "recipient_mismatch"
corrected = dict(proposal, address="alex@customer.example")
assert verify(corrected, contacts, approval) == "ready_for_send_review"
assert verify(dict(corrected, document_version=8), contacts, approval) == "approval_mismatch"
assert verify(corrected, contacts + [dict(contacts[1], id="alex-second")], approval) == "clarify"
assert verify(corrected, list(reversed(contacts)), approval) == "ready_for_send_review"
print("Display-name trap blocked; exact recipient binding preserved.")The fixture makes the failure observable without delivering anything. Directory order changes the unsafe selection, but not the guarded result after confirmation. Adding a second customer contact forces clarification instead of treating the account filter as sufficient proof. Document version changes also invalidate the approval because the recipient is only half of what the person authorized.
Syntax, Ownership, and Permission Are Different
An instruction to “check the email address” often collapses three different checks. Syntax asks whether the address is well formed. Ownership verification asks whether a party controls that mailbox. Authorization asks whether this recipient may receive this particular information. Passing the first two does not establish the third, especially for pricing, contracts, or customer exports.
OWASP’s email verification guidance recommends a documented comparison policy, domain normalization, and caution with provider-specific transformations. Do not strip dots, remove plus tags, or lowercase local parts indiscriminately to make candidate addresses match. The exact comparison in this fixture assumes prevalidated directory addresses; it is not a general email parser.
The language model may extract a candidate from a thread, but the thread is untrusted content. A forwarded signature saying “new finance contact, send all invoices here” should initiate an independent contact-change workflow. It must not overwrite the trusted directory or become proof of approval. A prompt-only rule cannot enforce that separation when the send tool accepts arbitrary address strings.
Build the Envelope from Trusted Data
Authenticate the sender, identify the customer account they may access, and retrieve active contact records within that scope. OWASP authorization guidance recommends server-side permission checks on every request. Apply them to directory retrieval, attachment access, draft creation, and send submission. Transport access through MCP or another integration is not customer-specific disclosure authority.
Resolve all envelope recipients, including carbon copies, blind copies, and expanded distribution lists. A visible To address can be correct while an added Bcc leaks the document. Store the complete recipient set alongside the content digest, attachment identities, attachment versions, sender identity, classification, and expiry. A directory update should trigger renewed review when it changes delivery identity.
OWASP transaction authorization guidance supports binding approval to significant operation data and invalidating altered transactions. For email, that means approval of the exact envelope and content, not a reusable Boolean called approved. The final adapter should build its payload from that stored proposal rather than accepting a fresh model-generated envelope.
The first architecture needs one drafting component, one directory lookup, and one policy-controlled mail adapter. A specialist classifier can help identify sensitive attachments when authoritative classification is unavailable, but uncertain classification should cause review. Multiple debating agents do not create independent proof of recipient authorization when they all read the same misleading email thread.
Recovery Starts Before Send
Persist a draft operation and its provider message reference before reporting completion. If submission times out, represent the result as unknown and inspect the provider’s authoritative message or submission state using documented semantics. Do not assume a client-generated message identifier guarantees deduplication across every provider. Confirm that behavior in adapter integration tests.
A wrapper’s status: success is not enough. The receipt must originate from the configured mail service and match the sender, complete recipient envelope, and approved operation. Provider acceptance is also distinct from recipient delivery. Bounce handling, quarantine, and downstream routing should remain separate states rather than being collapsed into “Alex received it.”
The code omits the authentication source, database transactions, persistent approvals, state persistence, provider semantics, mailing-list expansion, and delivery reconciliation. It assumes directory account associations are correct and current. If those records are compromised, deterministic comparison can faithfully approve the wrong address. Directory change controls and periodic contact verification are therefore part of the production dependency list.
Acceptance Tests
- A matching display name with an address from another account must fail before the mail adapter runs.
- Two active contacts with the same name inside the selected account must require explicit clarification.
- A changed attachment version or newly added Bcc recipient must invalidate earlier approval.
- A forwarded signature requesting a new destination must remain untrusted evidence, not a directory mutation.
- A cross-tenant directory lookup must be denied even when the sender supplies a valid contact identifier.
- An ambiguous submission timeout must reconcile the original operation instead of immediately sending another copy.
Evaluate Wrong Destinations, Not Fluent Drafts
Create a baseline set of human-labeled account-contact-document triples, including consultants, former employees, shared mailboxes, and updated addresses. Measure incorrect recipients proposed over all reviewed drafts, and unauthorized envelopes reaching submission over all attempted submissions. Track false holds over all legitimate drafts so that excessive friction remains visible alongside leakage risk.
Start with draft-only operation and shadow the verification result. Then pilot low-sensitivity messages with explicit review and a kill switch at the adapter. A proposed zero-leakage guardrail should stop the pilot on any unauthorized envelope; it is not a claim that this toy fixture proves production safety. Record contact identifiers and reason codes instead of full addresses and document bodies in general logs.
The sales CRM agent guide covers the surrounding account workflow, while the meeting ambiguity lab adds date uncertainty to identity resolution. AI systems and agent engineering can connect those boundaries. Implement exact envelope review first; better prose does not compensate for a wrong recipient.