Building practical AI
Deepak Kumar Tiwari

Deepak Kumar Tiwari

Technology & AI Leader · Building practical systems

Multimodal inspection / Guide 100

Scan-to-BIM Deviation Reports Need Survey Control and Coverage

Build a scan-to-BIM deviation harness with survey control, explicit IFC transforms, uncertainty intervals, and coverage before engineers approve discrepancy tickets.

Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.

Reference workflow: Survey and IFC revisions; Element correspondence; Controlled registration; Deviation and coverage harness; Engineer discrepancy review.
Reference workflow. Steps identify responsibilities, not necessarily separate agents. Branches and recovery paths are explained below. Open full-size diagram.

A hypothetical site scan appears to show every wall on a floor displaced by 80 millimeters. An assistant drafts dozens of installation defect tickets. The scan uses a survey coordinate frame while the IFC geometry was interpreted without its placement transform. The apparent construction problem is a comparison problem. Before generating any deviation report, the harness must establish common coordinates, correct elements, and sufficient observed surface coverage.

Raw inputs include scanner observations, registration records, survey control, an IFC revision, and supporting site photographs. A multimodal model can suggest likely element correspondences or summarize visible obstructions. Specialist survey and geometry tools compute measurements. Their output remains conditional on registration, representation, and sampling assumptions; it is not infallible physical ground truth or automatic acceptance of construction work.

Preserve the Survey Chain

Record project, survey session, scan station identifiers, scanner configuration, calibration reference, capture intervals, control network revision, and source digests. Attach the IFC file digest, schema version, issue purpose, and approved design revision. A coordination model and an issued construction model may represent different obligations. A scan taken before a wall was finished should not be evaluated as if it documented final installation.

Define a right-handed project frame, axis directions, elevation datum, and length units. Preserve the raw scanner frame and every transformation into project coordinates. Use meters for survey coordinates and explicitly convert reported deviations to millimeters. Large map coordinates may require a local origin for numerical stability, but that local offset must be reversible and recorded rather than silently discarded.

IfcOpenShell’s geometry processing documentation describes local vertices, placement matrices, geometry representations, and unit conversion. Applying a placement twice is as wrong as omitting it. Geometry extraction settings, opening treatment, tessellation precision, and kernel version should be part of the evidence manifest. An element identifier is useful only within the correct model and revision context.

Register Independently of the Suspected Defect

Open3D’s ICP registration tutorial describes iterative refinement from an initial alignment. A site with repeated corridors or parallel walls can produce plausible but incorrect correspondences. Validate against independent survey control and check points, not just the surfaces being tested. Otherwise, fitting the scan to a displaced wall can reduce the very deviation the inspection is meant to expose.

Maintain separate residuals for survey control, registration, and element comparison. A high registration error should generate a survey review request, not hundreds of wall defects. An apparently low residual still requires identity and spatial distribution checks. Measurements concentrated near one corner may leave rotation weakly constrained across the rest of the floor.

For photographs, retain camera identity, image digest, capture time, and top-left pixel coordinates for annotations. They can explain temporary partitions or occlusion but should not supply physical dimensions unless independently calibrated and registered. For moving scan or video acquisition, preserve timestamp-to-pose interpolation and its uncertainty. Repeated frames or dense returns from one viewpoint are correlated rather than independent confirmations.

Measure the Intended Surface

Choose the comparison property explicitly: signed wall-normal offset, verticality, clearance, or point-to-surface distance. A nearest surface may belong to furniture rather than the wall. Match element semantics and expected region before measuring. Temporary objects should be classified as obstruction or correspondence uncertainty, not automatically treated as installed building geometry.

Open3D’s distance query documentation explains the assumptions behind signed distance and occupancy. Open or incomplete building surfaces may not define a valid inside and outside. Use a verified local surface normal or another approved measurement method when appropriate. Axis-aligned bounding box overlap is only broad-phase candidate filtering, never proof of real mesh collision or measured clearance.

A Controlled Deviation Fixture

This self-contained Python example tests the HARNESS on synthetic structured measurements and metadata. It does not test a trained vision model, camera, scanner, CAD engine, IFC parser, or actual survey performance. The ten millimeter review tolerance and three millimeter control budget are illustrative policy values, not construction standards or contractual tolerances.

from math import isfinite

inspection_plan = frozenset({1, 2, 3, 4})
base = dict(model="floor-r9", survey="scan-21", frame="project-mm",
            required_cells={1, 2, 3, 4}, observed_cells={1, 2, 3, 4},
            offset_mm=4.0, measurement_bound_mm=1.0, control_bound_mm=2.0)

def assess(row, required_cells):
    if (row["model"], row["survey"], row["frame"]) != (
            "floor-r9", "scan-21", "project-mm"):
        return "invalid-comparison"
    if not required_cells or row["required_cells"] != required_cells:
        return "coverage-plan-mismatch"
    if not required_cells <= row["observed_cells"]:
        return "coverage-unknown"
    d, m, c = (row["offset_mm"], row["measurement_bound_mm"], row["control_bound_mm"])
    if not all(isfinite(v) for v in (d, m, c)) or min(m, c) < 0:
        return "measurement-unknown"
    if c > 3:
        return "survey-review"
    bound = m + c
    if abs(d) - bound > 10:
        return "deviation-review"
    return "within-band-evidence" if abs(d) + bound <= 10 else "uncertain"

assert assess(base, inspection_plan) == "within-band-evidence"
assert assess(dict(base, offset_mm=18), inspection_plan) == "deviation-review"
assert assess(dict(base, offset_mm=9), inspection_plan) == "uncertain"
assert assess(dict(base, observed_cells={1, 2}), inspection_plan) == "coverage-unknown"
assert assess(dict(base, control_bound_mm=4), inspection_plan) == "survey-review"
assert assess(dict(base, frame="scanner-m"), inspection_plan) == "invalid-comparison"
assert assess(dict(base, observed_cells={1}), inspection_plan) == "coverage-unknown"
assert assess(dict(base, required_cells={1}, observed_cells={1}), inspection_plan) == "coverage-plan-mismatch"
assert assess(dict(base, required_cells=set()), inspection_plan) == "coverage-plan-mismatch"
assert assess(base, frozenset()) == "coverage-plan-mismatch"
assert inspection_plan == frozenset({1, 2, 3, 4})
for invalid in (float("nan"), float("inf"), float("-inf")):
    for key in ("offset_mm", "measurement_bound_mm", "control_bound_mm"):
        assert assess(dict(base, **{key: invalid}), inspection_plan) == "measurement-unknown"
print("synthetic survey assertions passed")

The additive uncertainty rule assumes defensible bounds and keeps control error distinct from feature measurement error. A surveyor must establish the real uncertainty budget, including instrument calibration, target localization, registration, representation approximation, and surface fitting. Statistical estimates require an appropriate propagation model; simply adding arbitrary model confidences would have no physical meaning.

Coverage cells are frozen independently before processing the scan and passed as the trusted inspection-plan argument. The result’s declared requirements must match that plan; deleting missing cells from the result cannot shrink the denominator. Production must resolve this argument from the approved versioned plan, never from the result being checked. Dense points on one visible patch cannot compensate for an unobserved upper wall. Unscanned space is unknown, not missing construction and not passed construction. The strict fixture requires every cell; a production plan may allow specific exclusions, but each exclusion needs reviewer authorization and must remain visible in the denominator.

Create a Bounded Discrepancy Ticket

An engineer reviews the element identity, control checks, transform chain, source points, selected surface, deviation interval, and coverage map. They may request a targeted rescan, a survey correction, or an installation investigation. The assistant cannot alter the design model, issue a contractual nonconformance, or authorize demolition. A geometric mismatch is a starting point for investigation, not an automatic finding of fault.

IFC or BIM exit counts and geometric clearances are not life-safety code certification. Qualified reviewers must set jurisdiction, occupancy, and applicable rules through the proper review process. Likewise, a close surface match cannot establish structural integrity, hidden reinforcement, fire performance, or safe occupancy. Those limitations belong in the packet whenever its scope could be misunderstood.

Apply server-side project and role authorization to scans and models. Embedded IFC properties and visible site signage are untrusted content, never instructions to change permissions. Approved tickets use project, element, model revision, survey session, and finding identity for deduplication. Reconcile timeout outcomes before retrying. New survey or model evidence invalidates stale conclusions without erasing prior reviewer decisions.

Acceptance Tests and Evaluation

  • Omit an IFC placement transform; independent survey checks invalidate the comparison before wall tickets are produced.
  • Apply meter coordinates as millimeters; a known control distance catches the scale error.
  • Hide half a wall behind temporary storage; retain those cells as unknown. Deleting them from the result’s declared requirements must produce a plan mismatch, not within-band evidence.
  • Register against a repeated corridor; independent check points expose the wrong spatial correspondence.
  • Increase control uncertainty beyond budget; route to survey review instead of installation blame.
  • Change the IFC revision during ticket approval; require a new comparison and preserve the old evidence lineage.

Baseline against surveyor-approved measurements and engineer-adjudicated element findings. Report confirmed discrepancies divided by all escalated discrepancies, detected adjudicated deviations divided by all adjudicated deviations, and observed required surface area divided by all required surface area. Track false installation tickets caused by registration faults separately. Evaluate by floor and survey session, not randomly separated points from one scan.

Begin with one floor, one element class, and a verified control network. Construction project workflows provide ownership after a discrepancy is established, while printed-part scan comparison explains the smaller-scale datum problem. A focused AI systems implementation should first make invalid comparisons impossible to mistake for installation defects.

Sources