Agent failure labs / Guide 64
Medical Intake Must Not Turn an Unanswered Question Into No
Reproduce a missing-answer failure in AI medical intake escalation, preserve patient statements, and verify a human handoff without making clinical decisions.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
A hypothetical patient leaves an intake question unanswered and asks to speak with a member of the care team. The assistant’s summary says “No additional concerns” because the form adapter replaced a missing answer with false. A downstream queue treats the intake as complete. Nobody made an explicit diagnosis, yet an information-handling shortcut has suppressed the requested human contact.
This lab is about intake completeness and escalation ownership only. It does not diagnose, recommend treatment, calculate clinical urgency, or define a universally validated triage rule. Local clinicians must approve the questionnaire, escalation pathways, service hours, and patient-facing language for their setting. The exercise demonstrates a software distinction: unknown is not no, and a requested handoff is not an acknowledged handoff.
Preserve What Was Actually Answered
HL7 FHIR R4 QuestionnaireResponse represents complete or partial questionnaire answers and records information about the subject, author, source, and time of capture. That provides a useful modeling foundation: the answer should remain associated with the actual question and respondent. It does not authorize an assistant to fill unanswered fields with reassuring defaults.
Keep the original patient statement alongside any extracted fields, with restricted access and retention. A blank field may mean skipped, interrupted, misunderstood, or unavailable. Those possibilities require different follow-up, but none is equivalent to a negative answer. The application should display the missingness explicitly rather than producing a polished summary that hides it.
The fixture below contains a missing answer and an explicit request for staff contact. The naive function ignores the request and coerces the blank value into false. The fixed function emits a staff-review packet with a reason and pending acknowledgment. Even the complete comparison fixture remains an intake packet awaiting review, not a declaration that the patient is safe.
from copy import deepcopy
fixture = {"intake": "I-17", "revision": 3, "questionnaire": "local-v2",
"answers": {"contact_requested": True, "concern_reported": None},
"source": "patient_form", "statement": "Please have staff contact me."}
def naive(item):
return "complete_no_concerns" if not bool(item["answers"].get("concern_reported")) else "review"
def guarded(item, acknowledged):
answers = item["answers"]
reasons = []
if item["questionnaire"] != "local-v2":
reasons.append("questionnaire_version")
if any(type(answers.get(k)) is not bool
for k in ("contact_requested", "concern_reported")):
reasons.append("incomplete_or_invalid_answer")
if answers.get("contact_requested") is True:
reasons.append("staff_contact_requested")
if answers.get("concern_reported") is True:
reasons.append("reported_concern_for_staff_review")
key = (item["intake"], item["revision"])
return {"route": "staff_review" if reasons else "intake_review",
"reasons": reasons, "statement": item["statement"],
"handoff": "acknowledged" if key in acknowledged else "pending",
"clinical_decision": None}
before = deepcopy(fixture)
assert naive(fixture) == "complete_no_concerns"
packet = guarded(fixture, set())
assert packet["route"] == "staff_review"
assert "incomplete_or_invalid_answer" in packet["reasons"]
assert packet["handoff"] == "pending" and packet["clinical_decision"] is None
assert guarded(fixture, {("I-17", 2)})["handoff"] == "pending"
assert guarded(fixture, {("I-17", 3)})["handoff"] == "acknowledged"
complete = dict(fixture, answers={"contact_requested": False, "concern_reported": False})
assert guarded(complete, set())["route"] == "intake_review"
assert fixture == before and packet["statement"] == fixture["statement"]This is an explicitly synthetic deterministic simulation, not a real model experiment, clinical evaluation, or test of a medical triage system. Its routing labels are illustrative administrative states. No patient record is changed, no notification is sent, and no clinical action occurs. The acknowledgment set is trusted fixture data, not evidence that a real professional has accepted responsibility.
Do Not Solve Missingness With a Prompt
A prompt instructing the assistant to “be cautious” cannot recover a blank answer that an adapter already converted to false. The decisive guard belongs before summarization, where the application can distinguish absent, invalid, and explicitly negative responses. Summaries should be derived from that typed representation and should retain uncertainty rather than silently completing the form.
Likewise, an assistant cannot create available clinical staff by promising a callback. FHIR R4 Task separates workflow state from the operational details of the work and supports tracking ownership and completion. An intake packet can therefore exist without implying that staff have read it. Received, accepted, in progress, and completed must have locally defined meanings and authoritative transitions.
The minimum architecture uses a versioned questionnaire, an intake repository, a deterministic completeness check, and a staffed review queue. A language model may help draft a source-linked summary for staff. A translation specialist may be useful for supported languages, but unresolved translation must remain visible and must not delay an established safety pathway. Additional agent votes do not validate clinical urgency.
Design the Handoff as an Owned Obligation
The organization needs an explicit fallback when its queue is unavailable or unstaffed. Clinicians and operations owners define that pathway, including approved emergency-access information for the jurisdiction and circumstances. The assistant must not reassure a person that waiting is safe merely because the packet was created. Avoid making access to established human help contingent on completing every form field.
WHO’s guidance on AI for health emphasizes ethics, human rights, accountability, and responsiveness to affected communities and healthcare workers. Here, the engineering application is narrow: disclose the system’s role, preserve patient agency, and make an actual human handoff observable. The guidance does not clinically validate this toy checklist or its administrative routing choices.
Production dependencies omitted include the authentication source, patient-to-encounter matching, database transactions, provider semantics, clinical approvals, and durable state persistence. Verify patient or authorized representative identity using the approved server-side workflow. Enforce tenant and care-team access before retrieving records. Do not let an uploaded document instruct the tool to attach another patient’s chart or mark a task complete.
An acknowledgment must originate from the authorized staff workflow and match intake identity and revision. A returned status: success from a notification connector only establishes whatever that connector actually acknowledges, not clinical receipt. If submission times out, record the outcome as unknown, reconcile the same task identifier, and follow the approved fallback. Blindly resending creates competing tasks without resolving ownership.
Acceptance Tests
- A missing answer must reproduce naive false completion and produce an explicit incomplete-answer reason in the guarded packet.
- An explicit staff-contact request must remain visible even when every other answer is negative.
- A string such as “false” must be treated as invalid typed input, not converted through generic truthiness.
- An acknowledgment for an earlier intake revision must not acknowledge the patient’s later amendment.
- An unsupported questionnaire version must enter review rather than silently reuse an older completeness definition.
- Queue unavailability must activate the locally approved fallback without claiming delivery, clinical clearance, or a guaranteed response time.
Measure Information Loss, Not Diagnostic Accuracy
Start with synthetic forms and clinician-reviewed administrative expectations. The baseline is the coercing adapter. Count falsely complete packets divided by packets containing missing or invalid answers. Track omitted contact requests over all forms explicitly requesting staff contact. These are information-integrity metrics; calling them clinical sensitivity would misrepresent what the harness measures.
During a supervised pilot, measure unacknowledged handoffs over all packets requiring staff review, including outages and after-hours arrivals. Audit language accessibility, correction burden, and source fidelity with approved privacy controls. Promotion requires local clinical governance and operational capacity, not merely passing assertions. No universal triage safety or patient-outcome improvement is claimed.
The medical record summarization guide discusses provenance in broader records. The tool argument lab examines typed boundary failures. For AI systems engineering, the first implementation should remove false defaults and expose pending ownership. Preserving one unanswered question is more valuable than making an unsupported clinical claim sound certain.