Loan Assistants Must Not Promote Draft Text Into a Decision

An employee asks a hypothetical loan assistant to summarize a complete application for an authorized reviewer. The assistant returns an object containing document notes and an extra field: decision: approved. A generic update helper merges the whole object into the application record. The dashboard now displays approval even though no human made a credit decision.

This lab tests authority leakage through mass assignment. It does not build an underwriting model, infer creditworthiness, rank applicants, or automate approval, denial, pricing, or credit limits. The fixed behavior produces an evidence packet for a named human workflow and leaves decision state untouched. The unsafe branch exists only as an in-memory mock demonstrating what the integration must prohibit.

Distinguish Preparation From Decision Authority

Application completeness, evidence accuracy, and a lending decision are different concepts. A complete packet may contain contradictory or outdated evidence. A reviewer must be able to inspect original sources, correct extraction, and apply the institution’s approved process. Do not encode “all documents present” as a hidden approval recommendation or use document gaps as automatic denial criteria.

OWASP’s authorization guidance distinguishes authentication from authorization and recommends checking permissions for each request. An assistant permitted to prepare documents should not inherit an employee’s separate decision permissions. Tool credentials should expose only the preparation operation, with applicant and tenant scoping enforced on the server.

The fixture deliberately contains an unauthorized decision field. The naive implementation performs a dictionary merge, reproducing a local status change. The guarded implementation accepts only the preparation contract, checks references against an allowed evidence set, and rejects additional fields. A corrected packet can be prepared, but even its successful result contains no lending recommendation.

from copy import deepcopy

application = {"id": "L-42", "tenant": "T1", "revision": 6,
               "decision": "pending_human_review"}
draft = {"application": "L-42", "revision": 6, "evidence": ["doc-1"],
         "missing": [], "decision": "approved"}

def naive(record, candidate):
    return dict(record, **candidate)

def guarded(record, candidate, actor_tenant, allowed_evidence):
    required = {"application", "revision", "evidence", "missing"}
    if actor_tenant != record["tenant"]:
        return "denied", None
    if set(candidate) != required:
        return "invalid_fields", None
    if candidate["application"] != record["id"] or candidate["revision"] != record["revision"]:
        return "refresh_required", None
    if not all(type(candidate[k]) is list for k in ("evidence", "missing")):
        return "invalid_shape", None
    if not all(type(x) is str and x in allowed_evidence for x in candidate["evidence"]):
        return "unverified_reference", None
    if not all(type(x) is str for x in candidate["missing"]):
        return "invalid_shape", None
    return "packet_prepared", deepcopy(candidate)

before = deepcopy(application)
assert naive(application, draft)["decision"] == "approved"
assert guarded(application, draft, "T1", {"doc-1"}) == ("invalid_fields", None)
clean = {k: v for k, v in draft.items() if k != "decision"}
status, packet = guarded(application, clean, "T1", {"doc-1"})
assert status == "packet_prepared" and "decision" not in packet
assert guarded(application, clean, "T2", {"doc-1"})[0] == "denied"
assert guarded(application, dict(clean, revision=5), "T1", {"doc-1"})[0] == "refresh_required"
assert guarded(application, dict(clean, evidence=["other-file"]), "T1", {"doc-1"})[0] == "unverified_reference"
assert application == before and application["decision"] == "pending_human_review"

This is an explicitly synthetic deterministic simulation of a possible integration failure, not a real model experiment or credit assessment. No applicant is evaluated and no decision, notice, payment, or external update occurs. The mock approved value demonstrates a forbidden state transition rather than recommending an automated lending rule.

Close the Schema and the Capability

JSON Schema’s object reference explains that declaring properties does not make them required and that additional properties are allowed by default. A preparation schema therefore needs both required fields and an explicit rule rejecting extras. The Python block implements a tiny equivalent check for its own contract; it is not a general JSON Schema validator.

Rejecting a decision field is stronger than silently stripping it. The rejection reveals that a producer is attempting an unsupported operation, allowing maintainers to inspect the contract mismatch. Return a bounded error describing allowed preparation fields without exposing internal lending rules. Do not ask the model to try alternate names until one slips through a permissive update helper.

Schema validation alone is insufficient. A tool accepting only notes could still attach text saying “approved” where the UI presents notes as an official decision. Render preparation content in a clearly labeled evidence area. Official application status must come exclusively from the authorized decision system, with origin and revision checks. The model never supplies reviewer identity or a decision receipt.

A specialized document extractor can help handle different statement formats. It should expose source spans and uncertainty, not an applicant risk score. A policy retriever can locate approved instructions for staff without deciding how they apply to a person. More autonomous agents would expand the information surface without resolving who has authority to decide.

Preserve Review and Applicant Protections

The FTC’s guidance on consumer reports and credit notices describes notice obligations when consumer-report information contributes to adverse action or risk-based pricing. It is United States guidance, not a universal lending procedure. Compliance counsel must confirm current requirements for the institution, product, jurisdiction, and decision type; this article does not implement those notices.

An assistant must not invent reasons for a human decision or convert missing documents into an adverse-action explanation. The approved decision workflow records the actual reasons and controls any required communication. Keep a correction and human review route available. Prevent administrative automation from creating indefinite delays that function like unrecorded denials.

Fairness review must include packet preparation, not only final outcomes. Audit extraction errors, missing-document flags, and review delays across lawfully evaluated groups using an approved process. Do not infer protected attributes from names, photographs, addresses, accents, or social content. Removing explicit protected fields does not prove the workflow is fair, and a human reviewer does not automatically eliminate bias.

The snippet omits authentication sources, database transactions, provider semantics, approval persistence, document freshness verification, and durable state. Store evidence access permissions separately from model output. Tenant filtering occurs before retrieval. Uploaded instructions claiming to be an executive approval remain untrusted content. Logs should record contract violations and operation identifiers without reproducing financial documents or personal details.

Acceptance Tests

• The naive merge must reproduce a mock approval from the same candidate that the fixed boundary rejects.
• A valid preparation packet must never change approval, denial, pricing, limit, or applicant ranking fields.
• Unknown evidence identifiers must be rejected even if the candidate contains persuasive summaries or high confidence.
• An outdated application revision must require a refreshed packet, not silently overwrite later corrections.
• A user from another tenant must receive neither application content nor evidence-reference confirmation.
• Connector timeouts must preserve unknown delivery status; retries must reconcile the same packet identifier rather than create a second review case.

Evaluate Boundary Violations

Build a synthetic suite of extra fields, nested decision attempts, stale packets, invented evidence, and notes that impersonate official status. Compare the permissive merge with the closed preparation endpoint. Count unauthorized decision-state changes over all attempted preparation requests, and count false evidence citations over all citations presented. Neither metric measures lending quality or fairness by itself.

Start in a read-only staff pilot with every packet inspected. Track correction time, missed source discrepancies, and queue age, retaining unresolved cases in the denominator. Proposed expansion requires zero decision writes from preparation credentials and a reviewed applicant correction path. These are requirements, not deployment results.

The home loan application guide covers document orchestration, while the tool argument lab examines generic schema bypasses. AI systems engineering should first remove the generic application update tool from the assistant. A useful preparation system has no need to possess a lending decision capability.

Sources

• OWASP: Authorization Cheat Sheet
• JSON Schema: Object validation
• FTC: Consumer reports and credit notices