Multimodal inspection / Guide 78
Verify Construction Photos Against the Right Drawing
Verify construction photos against approved drawing revisions using element identity, surveyed coordinates, coverage checks, and explicit measurement uncertainty.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
A hypothetical site engineer uploads a photograph of a newly installed doorway and asks whether it matches drawing A-204. The assistant sees a doorway in roughly the expected location and answers yes. The drawing is revision C, however, while the approved issue is revision D. A stack of materials hides the lower jamb, and the photograph has no surveyed reference tying pixels to the drawing’s coordinate system.
The useful response is not a confident visual match. It is a packet that identifies the probable element, the drawing revision, visible coverage, registration quality, and measurements still needed. This workflow supports review and requests for information. It does not certify construction compliance, structural adequacy, or concealed workmanship from photographs.
Lock the Reference Before Comparing
Resolve project, building, level, grid, element identifier, and approved drawing issue through the document-control system. Filenames such as final or latest are insufficient. Preserve the drawing digest, issue date, approval status, and the photo capture event. A later drawing should not silently rewrite the reference used for an earlier inspection; it creates a new comparison with its own review history.
For each photograph, retain original bytes, capture time, uploader, camera configuration when available, frame identifier, and transformations such as crop or rotation. W3C PROV models derivation and responsibility across processing steps. That helps distinguish a candidate found in a cropped image from the source frame a surveyor actually used, without claiming that provenance alone proves accurate capture.
The construction project guide covers document coordination and follow-up. This lab is narrower: it checks whether one proposed geometric comparison has enough identity and measurement evidence to enter human review. The BIM exit review lab demonstrates why even well-identified geometry cannot replace a qualified interpretation of safety requirements.
Separate Appearance From Geometry
A model may suggest that the doorway corresponds to element D-12. That is candidate interpretation. A specialist must confirm the association and establish how image coordinates relate to the project reference. A two-dimensional drawing elevation, a floor plan, and a perspective photograph do not share coordinates merely because they depict the same object.
OpenCV’s calibration tutorial explains camera intrinsics, distortion, and extrinsic pose. A practical comparison also needs registration to surveyed control, known units, and uncertainty propagation. A planar homography may suit a genuinely planar surface under its assumptions; it is not a general solution for protruding objects or an entire three-dimensional room.
Record the project origin, axis directions, elevation datum, and millimeter units. Keep image coordinates in pixels until a validated transformation is applied. Estimate uncertainty from survey control, calibration, feature localization, and registration assumptions rather than treating a low model uncertainty score as a metrology result. RGB imagery cannot reveal reinforcement inside concrete or establish structural integrity behind a finished wall.
A Synthetic Deviation Harness
The example tests the HARNESS using structured synthetic survey results, not actual photographs, a vision model, or a CAD engine. It compares one element’s lateral deviation against an illustrative ten-millimeter policy boundary. That boundary is not a construction standard. Required jamb coverage, drawing identity, frame provenance, coordinate metadata, and uncertainty must be acceptable before the number is interpreted.
import math
sample = {"project": "site-8", "element": "D-12", "drawing": "A-204-D",
"photo": "capture-61", "photo_version": "original",
"coverage": {"left_jamb", "right_jamb", "head"},
"coords": "grid_B2_local", "units": "mm", "registration": "survey-9",
"deviation_mm": 14.0, "uncertainty_mm": 2.0}
def compare(record):
if (record["project"], record["element"], record["drawing"]) != (
"site-8", "D-12", "A-204-D"):
return "reference_mismatch"
if not {"left_jamb", "right_jamb", "head"} <= record["coverage"]:
return "coverage_unknown"
if (not record["photo"] or record["photo_version"] != "original"
or record["coords"] != "grid_B2_local" or record["units"] != "mm"
or not record["registration"]):
return "registration_unknown"
deviation, error = record["deviation_mm"], record["uncertainty_mm"]
if (not all(math.isfinite(x) for x in (deviation, error))
or not 0 <= error <= 3.0):
return "remeasure"
magnitude = abs(deviation)
if magnitude - error > 10.0:
return "deviation_review"
if magnitude + error >= 10.0:
return "remeasure"
return "within_example_band_review"
assert compare(sample) == "deviation_review"
assert compare(dict(sample, drawing="A-204-C")) == "reference_mismatch"
assert compare(dict(sample, coverage={"head"})) == "coverage_unknown"
assert compare(dict(sample, units="in")) == "registration_unknown"
assert compare(dict(sample, deviation_mm=9.0)) == "remeasure"
assert compare(dict(sample, uncertainty_mm=4.0)) == "remeasure"
assert compare(dict(sample, deviation_mm=2.0)) == "within_example_band_review"
print("synthetic drawing, coverage, registration, and interval checks passed")The lower uncertainty bound must exceed the example boundary before a deviation-review packet is generated. An interval crossing the boundary triggers remeasurement. A result within the example band still requires review and says nothing about other dimensions, door hardware, fire rating, or installation quality. Deterministic arithmetic cannot repair an incorrect element association or a falsely asserted survey reference.
Build a Bounded Review Packet
Present the approved drawing excerpt, original photograph, candidate element match, surveyed reference, measured dimension, uncertainty interval, and uncovered regions together. Avoid a green building-wide pass indicator. When a jamb is occluded, the packet should ask for that specific missing evidence rather than merely requesting “better photos.” Repeated frames from a walk-through video remain correlated and cannot substitute for independent viewpoints or survey control.
Use authenticated project access and tenant-scoped document storage. Contractors should not gain access to unrelated project drawings through a guessed element ID. Treat OCR notes and drawing annotations as data, not instructions to approve work. A specialist survey or geometry service earns its place by producing traceable measurements; a conversational agent guessing distances does not.
The sample omits image decoding, drawing parsing, identity resolution, survey instruments, real registration, calibration estimation, authorization, and persistence. Production geometry tools compute results under their own assumptions and tolerances; they are not infallible physical ground truth. Require a qualified reviewer to accept the comparison and decide whether an advisory issue or request for information is appropriate.
Persist approved tickets against the evidence revision. If a ticket write times out, reconcile its operation identity before retrying. A revised drawing can invalidate a pending comparison, but it should not erase the historical observation. Preserve the distinction between “no longer relevant to this issue” and “the original measurement was wrong,” because they imply different corrective actions.
Acceptance Tests
- Compare a current photograph with a superseded drawing and require reference resolution before any conformity statement.
- Occlude one required jamb and retain unknown coverage even if the remaining visible geometry appears aligned.
- Supply inch measurements under a millimeter expectation and block comparison rather than silently treating the values as compatible.
- Place the measurement interval across the illustrative boundary and request remeasurement instead of a binary verdict.
- Match a similar doorway on another level and require element identity confirmation before displaying a deviation.
- Revise the drawing while a ticket awaits approval and invalidate the pending review snapshot without deleting its history.
Evaluate by Element and Capture
Use independently surveyed elements as the comparison baseline. Measure correct element associations divided by all adjudicated associations, missed measured deviations divided by confirmed deviations within covered scope, and abstentions divided by all attempted element comparisons. Report unobservable elements separately rather than counting them as compliant. Split evaluation by project and capture session to avoid repeated-frame leakage.
NIST’s AI Risk Management Framework supports explicit evaluation and risk ownership, not certification of these example limits. Start with one drawing type and one measurable feature. The first AI systems engineering deliverable should be an evidence packet that refuses stale revisions and unknown registration. Only expand to additional dimensions after the survey and review process demonstrates that the numbers mean what the interface says.