Multimodal inspection / Guide 76
Car Photo Inspection Starts With Coverage, Not Confidence
Design car photo inspection coverage checks that preserve vehicle identity, missing views, and measurement uncertainty before a human reviews damage candidates.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
A hypothetical dealership receives six photographs of a returned car. Four show the front bumper from nearly identical angles, one shows the left doors, and one belongs to yesterday’s return. A vision assistant finds no obvious damage and drafts “vehicle condition verified.” The collection contains many images but little evidence about the roof, right side, rear bumper, or underside.
Car photo inspection should begin by identifying what was observed, not by averaging model confidence. The useful output is a bounded inspection packet listing visible candidates and missing views. It is not a roadworthiness decision, a repair authorization, or proof that hidden damage is absent. This article builds a synthetic harness for that packet boundary rather than a trained vision model.
Make Coverage an Explicit Contract
Define the intended scope before capture. A cosmetic exterior review might require front, rear, left, right, and roof views; another contract may require different panels or close-ups. The five-view set here is illustrative policy, not a vehicle inspection standard. A photographed panel can still be uninspectable because of glare, darkness, water, motion blur, or an obstruction.
Record vehicle identity from the authenticated return workflow, capture session, capture time, original media digest, camera identifier, and any crop or resize lineage. Do not rely solely on a filename or a plate inferred by the model. W3C PROV describes provenance through entities, activities, and responsible agents. That vocabulary helps connect an observation to the original photo and processing version without pretending provenance makes an image truthful.
Image coordinates should be explicit: pixel origin at the upper left, horizontal axis to the right, vertical axis downward, and dimensions after any rotation. Panel labels refer to the vehicle’s own left and right, not the viewer’s. Physical dimensions require a separately justified vehicle or panel coordinate system and measurement method. Store the transformation rather than silently converting a bounding box into millimeters.
Interpretation Is Not Measurement
A model may propose “possible scratch on left front door” with a source region. A specialist then checks the original image, verifies the panel identity, and decides whether another view or physical inspection is needed. Uncalibrated RGB photos do not establish dent depth, temperature, structural integrity, or hidden damage. A shiny reflection should not become a precise repair estimate.
OpenCV’s official calibration tutorial explains intrinsic parameters, distortion, extrinsic pose, and reprojection error. These support a measurement pipeline but do not make a single arbitrary photograph metrically reliable. A scale marker must be appropriate to the measured plane, and perspective, lens changes, and panel curvature can invalidate a simple pixel-to-length ratio.
The car dealership service guide handles customer and workshop coordination. This packet should feed that process as evidence for a service adviser, not create work orders by itself. The production-line harness applies a similar boundary to controlled captures, where calibration may be easier but coverage remains a separate requirement.
Test the Harness With Synthetic Records
The following stdlib example tests the HARNESS, not a camera, trained model, or real inspection performance. Records are synthetic structured measurements. A duplicate front view adds no coverage, a wrong vehicle blocks the packet, and excessive registration uncertainty requires recapture. Pixel uncertainty limits are illustrative local policy values.
import math
required = {"front", "rear", "left", "right", "roof"}
expected = ("vehicle-42", "return-3")
photos = [
{"vehicle": expected[0], "session": expected[1], "panel": panel,
"frame": "photo-" + panel, "frame_version": "original",
"coords": "image_xy_px", "registration_px": 0.6, "visible": True}
for panel in sorted(required)
]
def packet(records):
if any((r["vehicle"], r["session"]) != expected for r in records):
return {"state": "identity_mismatch", "coverage": 0.0}
usable = {r["panel"] for r in records
if r["visible"] and r["frame"]
and r["frame_version"] == "original"
and r["coords"] == "image_xy_px"
and math.isfinite(r["registration_px"])
and 0 <= r["registration_px"] <= 1.0}
covered = usable & required
return {"state": "human_review" if covered == required else "recapture",
"coverage": len(covered) / len(required),
"missing": sorted(required - covered)}
assert packet(photos)["state"] == "human_review"
partial = photos[:1] * 6
assert packet(partial)["coverage"] == 0.2
assert packet(partial)["state"] == "recapture"
wrong = [dict(photos[0], vehicle="vehicle-41")]
assert packet(wrong)["state"] == "identity_mismatch"
uncertain = [dict(p, registration_px=2.0) for p in photos]
assert packet(uncertain)["coverage"] == 0.0
assert packet([])["missing"] == sorted(required)
print("synthetic identity, coverage, and uncertainty checks passed")The permitted state is human review, never passed. Even perfect fixture coverage says nothing about defect recall. The example assumes panel labels and visibility flags were independently validated; a model hallucinating those inputs could still produce a misleading complete packet. Production must evaluate those upstream judgments separately and retain abstention when they cannot be established.
Bound the Human-Reviewed Action
Show the adviser a panel checklist, source photos, candidate regions, unresolved identity issues, and reasons for recapture. Keep each observation linked to the original frame rather than only a compressed montage. Repeated video frames are correlated and must not be counted as independent confirmations. A new angle can improve visibility; a hundred copies cannot reveal an occluded panel.
Authenticate uploaders and isolate return records by tenant and vehicle session. Treat OCR text, metadata, and embedded instructions as untrusted media content. The inspection worker should not have repair approval or customer messaging credentials. The example omits media decoding, identity verification, calibration estimation, secure storage, transaction handling, and actual ticket integration. A specialist measurement service is justified only when its method and uncertainty can be reviewed.
If a reviewer approves a recapture request or advisory ticket, persist the evidence version and operation identity. A timed-out ticket write remains unknown until the ticket system is queried; do not create duplicates. A later photo set creates a new packet revision rather than overwriting the original evidence. Retention rules should minimize incidental faces, plates, and location data while preserving required inspection records.
Acceptance Tests
- Supply six copies of one front view and verify that coverage remains one required view rather than six completed checks.
- Mix two vehicle sessions and block review readiness until identity is resolved through the return workflow.
- Mark the roof occluded and require recapture, even when every visible panel has a high no-defect score.
- Remove coordinate metadata or exceed registration uncertainty limits and prevent physical dimension claims.
- Propose structural clearance from RGB photographs and verify that the packet rejects that unsupported scope.
- Approve a ticket, lose acknowledgment, and reconcile the original operation without issuing another ticket automatically.
Evaluate by Vehicle Session
Compare the harness against an image-count baseline on independently reviewed return sessions. Measure complete usable sessions divided by all submitted sessions, wrong-vehicle packets admitted divided by mixed-identity sessions, and visible defect candidates missed divided by reviewer-confirmed visible defects. Report hidden or unobserved regions separately; they are not true negatives. Split evaluation by vehicle session so adjacent frames cannot leak between training and test sets.
NIST’s AI Risk Management Framework provides voluntary guidance for incorporating trustworthiness into design and evaluation, not a certification for this workflow. Start with advisory packets and mandatory adviser review. The first AI systems engineering step is a capture checklist that refuses to equate missing views with clean panels. Only then evaluate whether model suggestions save review time without obscuring uncertainty.