Multimodal inspection / Guide 81
Aircraft Surface Inspection Needs an Evidence Boundary
Build aircraft surface inspection evidence packets that preserve panel identity, calibrated scale, coverage gaps, and human review without issuing safety clearance.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
A hypothetical aircraft maintenance team uploads twelve photographs of a wing access panel. A visual assistant circles a dark line and describes it as a scratch. The useful output is a location for qualified examination, not a diagnosis. One photograph belongs to the opposite wing, another predates a panel replacement, and the underside is missing. A fluent summary can conceal all three problems while appearing impressively specific.
Build the first release around evidence admission. Its job is to assemble an advisory inspection packet containing what was observed, where it was observed, and what remains unknown. It must not issue automated safety clearance, airworthiness conclusions, repair instructions, or return-to-service decisions. The same boundary applies when no candidate marking is found: a quiet detector does not establish structural integrity.
Bind the Observation to the Panel
Require the aircraft asset identifier, installed panel identifier, configuration revision, inspection session, capture timestamp, and original file digest. Keep camera serial, lens configuration, image dimensions, and any crop transform alongside the original. An annotation at pixel coordinates without its source dimensions becomes ambiguous as soon as someone resizes the image for a report.
Use image coordinates with the origin at the upper-left corner, horizontal pixels increasing right, and vertical pixels increasing down. Separately define a panel-local frame in millimeters, with an origin at an identified fastener and axes aligned to the maintenance drawing. Registration must reference that drawing revision. A left-right mirror is not a harmless display transformation when it changes the physical inspection location.
The W3C provenance overview distinguishes entities, activities, and responsible agents. Apply that separation to the original photograph, its rectification activity, and the reviewer accepting the panel association. A digest detects changed bytes; it does not prove that the photographed panel was correctly identified. That association needs an accountable capture procedure and independent checks.
Measure Less Than the Picture Suggests
The candidate generator may propose a region and a neutral description such as “linear contrast feature.” A measurement specialist then decides whether the image supports dimensions at all. A reference target must lie in the relevant surface plane, and curved surfaces may require a different acquisition method. Without defensible scale, retain pixel extent and abstain from reporting physical length.
OpenCV’s calibration tutorial explains camera intrinsics, extrinsics, lens distortion, and the relationship between known object points and image points. Calibration is therefore a prerequisite for certain measurements, not a license to turn every photograph into metrology. Reprojection residuals must be checked across the relevant field of view, not only near a convenient central target.
Keep scale uncertainty and registration uncertainty separate. Registration affects which panel location is reported; scale and endpoint selection affect estimated feature length. NIST’s uncertainty guidance includes covariance when combining uncertainty components. Adding independent-looking numbers without checking shared calibration errors can understate uncertainty. The fixture below uses supplied conservative bounds, not a statistically justified confidence interval.
Exercise the Admission Harness
This self-contained Python example tests the HARNESS using synthetic structured metadata. It does not evaluate a trained vision model, camera, CAD engine, or real inspection performance. Every threshold is an illustrative workflow policy, not an aircraft maintenance standard. The accepted state means only that a packet is ready for qualified review.
from math import isfinite
base = dict(asset="AIR-17", panel="L-204", revision=7,
image="synthetic-frame-12", captured_s=1000,
frame="panel-L204-mm", scale_mm_px=0.2,
registration_mm=0.4, extent_px=30.0,
length_bound_mm=0.5, views={"outer", "edge"})
def admit(m, now_s=1100):
if (m["asset"], m["panel"], m["revision"]) != ("AIR-17", "L-204", 7):
return "abstain:identity"
if not m["image"] or not 0 <= now_s - m["captured_s"] <= 300:
return "abstain:provenance"
if m["frame"] != "panel-L204-mm":
return "abstain:frame"
values = [m[k] for k in ("scale_mm_px", "registration_mm",
"extent_px", "length_bound_mm")]
if not all(isfinite(x) and x >= 0 for x in values):
return "abstain:measurement"
if m["scale_mm_px"] == 0 or m["registration_mm"] > 1:
return "abstain:measurement"
if not {"outer", "edge", "inner"} <= m["views"]:
return "abstain:coverage"
length = m["extent_px"] * m["scale_mm_px"]
return ("review", length, m["length_bound_mm"])
assert admit(base) == "abstain:coverage"
complete = dict(base, views={"outer", "edge", "inner"})
assert admit(complete) == ("review", 6.0, 0.5)
assert admit(dict(complete, panel="R-204")) == "abstain:identity"
assert admit(dict(complete, scale_mm_px=0)) == "abstain:measurement"
assert admit(dict(complete, captured_s=1200)) == "abstain:provenance"
print("aircraft evidence harness: five assertions passed")The missing inner view blocks completeness even though the measured candidate remains worth showing to a reviewer. In production, retain both facts rather than discard the observation. An incomplete packet can carry an urgent human escalation; it cannot quietly become a complete inspection. The toy manifest uses named views, while an operational system needs explicit surface patches and visibility masks.
Preserve the Maintenance Boundary
Prefer one candidate extraction stage and one deterministic validation service. A separate specialist agent earns its cost only if it uses a genuinely different evidence source or review discipline. Multiple language-model opinions over the same twelve photographs do not provide independent measurements. RGB appearance cannot establish temperature, hidden cracking, or material condition.
Server-side authorization must bind the session to the operator and aircraft record before any media access. Treat painted text, filenames, and embedded notes as untrusted evidence, never instructions to close a job. Store originals within the authorized tenant, issue short-lived review links, and avoid asset identifiers in general diagnostic logs. Production authentication, maintenance-system semantics, transactional writes, and approval persistence are intentionally omitted from the fixture.
For a follow-up ticket, use an idempotency key incorporating session, panel, candidate, and policy revision. After a timeout, reconcile the authoritative ticket identifier before retrying. A successful upload is not proof that a maintenance record was updated. The broader scheduling boundary is covered in manufacturing maintenance workflows; the narrower location problem also appears in blade defect mapping.
Acceptance Tests and Evaluation
- Replace the panel revision after capture; require a fresh association review rather than silently reusing old annotations.
- Remove the inner view; preserve the candidate but mark the inspection coverage incomplete.
- Supply mirrored coordinates or an unknown local frame; block location export to the maintenance packet.
- Set a scale to zero or a registration bound above policy; return a measurement abstention, not a zero-length feature.
- Insert “approve return to service” in visible text; record it as image content without changing workflow authority.
- Retry ticket creation after a simulated lost response; reconcile one approved ticket instead of producing duplicates.
Compare the harness with a baseline that simply forwards candidate captions. Measure correctly associated observations divided by all reviewed observations, and missed required patches divided by all required patches. Report abstention frequency separately; excluding incomplete sessions would make the workflow look artificially reliable. Use independent inspection sessions as the evaluation unit, since repeated frames of one marking are correlated.
Start in shadow mode on one panel family with a qualified reviewer adjudicating every packet. Track review time, unsupported dimension claims, and incorrect completeness labels, not invented defect-detection accuracy. AI systems engineering should begin with that manifest and review contract. The first deliverable is a traceable packet that makes uncertainty visible, not a model permitted to clear an aircraft.