Agent failure labs / Guide 55
AI Shopping SKU Validation: The Cheapest Match Is the Wrong Pack
Reproduce an AI shopping SKU mismatch involving pack size and connector type, then validate exact product constraints, quote versions, and approval before checkout.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.

“Buy twelve USB-C charging cables, two meters long, from the approved supplier.” In this hypothetical purchasing request, search returns a cheap single cable, a twelve-pack of micro-USB cables, and a twelve-pack of the requested USB-C model. All three listings contain the words charging cable. Sorting by price chooses the single item and leaves eleven desks without equipment.
The operational failure is not that the assistant lacks shopping knowledge. It is that text similarity and sticker price were promoted into a purchase decision without checking units, connector, length, supplier, and quantity. This lab turns that mistake into a reproducible fixture. The output remains a candidate for review, never an actual checkout or payment.
Reproduce the Pack-Size Error
The unsafe selector filters by title and chooses the cheapest result. The guarded selector treats the requested product properties as hard constraints, checks available packs and catalog revision, and only then returns a unique SKU. Price is compared with a fixture budget after establishing how many physical items the quoted pack contains.
This is an explicitly synthetic deterministic simulation, not an actual model experiment, product recommendation, or merchant API test. Both implementations receive the same catalog and intent. No website is browsed by the snippet, no cart is created, and no payment occurs. All prices and limits are illustrative integers in USD minor units.
intent = {"units": 12, "connector": "USB-C", "length_cm": 200,
"supplier": "approved-1", "max_minor": 5000, "revision": 3}
catalog = [
{"sku": "single-c", "title": "Charging cable", "pack": 1, "connector": "USB-C",
"length_cm": 200, "supplier": "approved-1", "price": 600, "stock": 8, "revision": 3},
{"sku": "pack-micro", "title": "Charging cable", "pack": 12, "connector": "micro-USB",
"length_cm": 200, "supplier": "approved-1", "price": 2400, "stock": 4, "revision": 3},
{"sku": "pack-c", "title": "Charging cable", "pack": 12, "connector": "USB-C",
"length_cm": 200, "supplier": "approved-1", "price": 4200, "stock": 2, "revision": 3},
]
def naive(need, rows):
return min((r for r in rows if "cable" in r["title"].lower()), key=lambda r: r["price"])["sku"]
def guarded(need, rows):
matches = [r for r in rows if r["pack"] == need["units"]
and all(r[k] == need[k] for k in ("connector", "length_cm", "supplier", "revision"))
and r["stock"] >= 1 and 0 < r["price"] <= need["max_minor"]]
return matches[0]["sku"] if len(matches) == 1 else "review"
assert naive(intent, catalog) == "single-c"
assert guarded(intent, catalog) == "pack-c"
assert next(r for r in catalog if r["sku"] == "single-c")["pack"] != intent["units"]
assert guarded(intent, catalog[:2]) == "review"
assert guarded(dict(intent, revision=4), catalog) == "review"
assert guarded(dict(intent, max_minor=4000), catalog) == "review"
assert guarded(intent, [dict(r, stock=0) for r in catalog]) == "review"
print("Single-item bargain rejected; matching twelve-pack identified.")
The fixed rerun selects the correct pack on the original fixture. Removing that SKU does not make the next closest option acceptable; it yields review. This is an important contrast with ranking systems that always return a winner. The revision assertion also demonstrates that a previously valid catalog snapshot cannot be treated as a current quote by changing a date in the assistant’s explanation.
Convert Preferences into Explicit Constraints
Some shopping requirements are negotiable, such as color. Others define whether the product can perform the job. Ask the requester to distinguish those categories before searching. In this exercise, connector type, physical length, pack quantity, and approved supplier are hard constraints. The budget is an illustrative cap, not evidence that the catalog price is the final landed cost.
OWASP input validation guidance distinguishes syntactic validation from semantic validation within a business context. A well-formed SKU string is syntactically acceptable; a micro-USB SKU for a USB-C request is semantically wrong. Validate catalog feeds as untrusted input too, including types, unit labels, negative prices, missing revisions, and inconsistent pack descriptions.
A prompt saying “double-check compatibility” can help extract candidate attributes but cannot make missing supplier data true. Product descriptions may contain injected instructions or contradictory claims. Keep source fields, normalized values, and uncertainty separate. If connector data is absent, return unknown rather than infer it from a photograph or from similar products elsewhere in the catalog.
Bind the Quote, Not Just the Product Name
Use the authenticated buyer’s organization to select supplier contracts and purchasing privileges. OWASP authorization guidance recommends checking permission on each request and denying access by default. A shared shopping assistant must not retrieve another tenant’s negotiated prices or place orders against a budget merely because the requester knows its identifier.
After validating a SKU, obtain a current quote with currency, quantity, pack definition, shipping, taxes, delivery window, and substitution terms. Recompute the total from trusted fields. The fixture assumes one exact twelve-pack; supporting twelve singles or mixed pack sizes requires a separate quantity calculation and explicit approval, not silently broadening this predicate.
Store a proposal binding supplier, SKU, quantity, quote revision, total, shipping destination, and expiry. OWASP transaction authorization guidance recommends showing significant transaction data and preventing post-approval changes. If a supplier replaces the selected SKU during checkout, invalidate the proposal even when the replacement has the same title and price.
One retrieval component and a deterministic catalog validator are enough for this initial workflow. A specialist compatibility agent can earn its cost when it searches manufacturer documentation for complex electrical or mechanical constraints. Its findings still need source provenance and a review path; it should not override the hard constraints or convert a compatibility guess into purchasing authority.
Recover from Inventory and Checkout Races
Stock displayed during search is not a reservation. Revalidate at the order boundary, and handle changed stock or price by presenting a new quote. Avoid a fallback that buys from an unapproved supplier merely because the original supplier became unavailable. An unavailable matching SKU is a legitimate stopping state, not an instruction to relax requirements.
If order submission times out, record an unknown outcome and reconcile the original operation using the merchant’s documented order lookup and idempotency behavior. A returned status: success alone does not prove an order exists for the approved SKU and quantity. Match the authoritative merchant account, order identifier, line items, amount, and destination before reporting placement.
The snippet omits the authentication source, database transactions, persistent approvals, state persistence, provider semantics, stock reservation, checkout, and payment. It assumes catalog fields are already typed and accurate. Production validation must handle malformed records and external uncertainty; deterministic matching cannot guarantee physical compatibility or fulfillment when supplier information is incomplete or false.
Acceptance Tests
- A cheap single cable must fail a one-pack request for twelve physical items, despite an identical title.
- A twelve-pack with the wrong connector must be excluded even when it fits the budget.
- Missing compatibility evidence must return review rather than a guessed attribute value.
- A changed quote revision, shipping cost, or substitution must invalidate the approved order proposal.
- A supplier outside the authenticated tenant’s contract scope must be unavailable to both retrieval and checkout.
- A checkout timeout must reconcile the original order before any replacement attempt is authorized.
Measure Fulfillment-Relevant Errors
Build a baseline of requests whose required attributes have been labeled by purchasing staff. Include bundles, accessories, regional connectors, and inconsistent units. Measure incompatible selections over all reviewed selections, and correct abstentions over all requests with no valid catalog option. Track unnecessary holds over valid requests to expose overly rigid rules.
Begin in recommendation-only mode, then permit approved quotes for a narrow commodity category. The proposed guardrail is zero unapproved substitutions reaching checkout. Monitor received item counts and mismatch returns separately from search accuracy, because fulfillment can fail after a correct selection. These are evaluation recommendations, not measured results from a real purchasing deployment.
The personalized shopping assistant guide addresses the broader discovery journey. The agent spending limits lab explains why a valid product can still exceed aggregate authority. Backend and API engineering can enforce quote bindings. Start with pack size and compatibility, not with a more elaborate recommendation prompt.