Warehouse Pallet Vision Verification Needs More Than a Box

A hypothetical warehouse camera sees a wrapped pallet entering a bay. The assistant draws a rectangle around the load and declares that it fits. The rectangle excludes a rear carton hidden by the wrap, the bay drawing uses centimeters while the estimate uses millimeters, and the visible label belongs to a pallet that passed through earlier. A convincing overlay has concealed three independent reasons to abstain.

Build an advisory verification packet that checks load identity, dimensional evidence, coordinate conventions, and observed surfaces. It must not direct a forklift, approve a lift, or certify load stability from a picture. RGB appearance does not establish weight distribution, internal damage, temperature, or safe handling. Physical operations remain governed by the warehouse’s approved procedures and trained personnel.

Join the Load to the Capture Session

Require warehouse, bay, load identifier, pallet specification revision, capture session, source frame references, and acquisition timestamps. Bind a barcode read to the same physical tracking interval as the dimension measurement. Reading a valid barcode somewhere in the clip is not enough; another pallet may have crossed the camera between the scan and the selected frame.

Use image coordinates in pixels from the upper-left corner. Define the pallet frame in millimeters, with an origin at the front-left base corner, x across width, y toward the rear, and z upward. Define the bay frame independently and retain the registered transform between them. Unit conversion must be explicit before comparing either envelope with a specification.

The W3C provenance overview distinguishes entities and derivation activities. Use that distinction for the label read, video frame, depth measurement, load association, and reviewer decision. If an operator corrects the load identifier, invalidate any task draft tied to the earlier association instead of merely changing the label on its image.

Use Geometry Within Its Actual Scope

The candidate stage can propose a load silhouette or possible overhang. A specialist dimensional stage should provide calibrated measurements with visibility and uncertainty. OpenCV’s calibration documentation explains camera parameters, distortion, and object-to-image relationships. Those concepts support measurement, but an uncalibrated monocular rectangle is not a three-dimensional load envelope.

Even a measured axis-aligned bounding box is only a broad-phase geometry representation. Overlap is not a full contact or collision determination, and absence of overlap under uncertain registration does not establish operational clearance. Real narrow-phase checks need appropriate geometry and contact tools, with assumptions reviewed for the actual movement. The example below does not implement collision detection at all.

Carry dimensional uncertainty separately from pallet-to-bay registration uncertainty. NIST’s combined uncertainty guidance includes covariance and systematic effects. Shared calibration errors affect multiple dimensions together; simply collecting more frames cannot be assumed to eliminate them. A reviewer should see the bound used in each advisory comparison.

Test the Overhang Evidence Gate

This self-contained Python fixture tests the HARNESS with synthetic structured measurements. It does not evaluate a trained vision model, camera, CAD engine, or real inspection performance. The overhang and registration limits are illustrative workflow policy values, not handling standards. No real media processing or engine API calls are made.

from math import isfinite

base = dict(load="LOAD-22", spec=3, bay="B4", source="synthetic-frame",
            frame="pallet-front-left-mm", unit="mm", captured_s=100,
            scan_s=101, overhang_mm=8.0, bound_mm=4.0,
            registration_mm=2.0, sides={"front", "left", "right"})

def verify(m, now_s=120):
    if (m["load"], m["spec"], m["bay"]) != ("LOAD-22", 3, "B4"):
        return "abstain:identity"
    if m["unit"] != "mm" or m["frame"] != "pallet-front-left-mm":
        return "abstain:coordinates"
    if (not m["source"] or not all(isfinite(t) for t in (now_s, m["captured_s"]))
            or not 0 <= now_s - m["captured_s"] <= 30):
        return "abstain:provenance"
    if not isfinite(m["scan_s"]) or abs(m["scan_s"] - m["captured_s"]) > 2:
        return "abstain:scan-pair"
    if not all(isfinite(m[k]) and m[k] >= 0 for k in
               ("overhang_mm", "bound_mm", "registration_mm")):
        return "abstain:measurement"
    if m["registration_mm"] > 5:
        return "abstain:registration"
    if not {"front", "rear", "left", "right"} <= m["sides"]:
        return "recapture:rear-or-side"
    return "review-overhang" if m["overhang_mm"] + m["bound_mm"] > 10 else "review-evidence"

assert verify(base) == "recapture:rear-or-side"
complete = dict(base, sides={"front", "rear", "left", "right"})
assert verify(complete) == "review-overhang"
assert verify(dict(complete, unit="cm")) == "abstain:coordinates"
assert verify(dict(complete, scan_s=110)) == "abstain:scan-pair"
assert verify(dict(complete, overhang_mm=3)) == "review-evidence"
for invalid in (float("nan"), float("inf"), float("-inf")):
    assert verify(dict(complete, captured_s=invalid)) == "abstain:provenance"
    assert verify(complete, now_s=invalid) == "abstain:provenance"
    assert verify(dict(complete, scan_s=invalid)) == "abstain:scan-pair"
    assert verify(dict(complete, captured_s=invalid, scan_s=invalid), now_s=invalid) == "abstain:provenance"
assert verify(complete, now_s=100) == "review-overhang"
assert verify(complete, now_s=130) == "review-overhang"
assert verify(complete, now_s=131) == "abstain:provenance"
assert verify(dict(complete, scan_s=98)) == "review-overhang"
assert verify(dict(complete, scan_s=102)) == "review-overhang"
print("pallet evidence harness: synthetic assertions passed")

The central overhang estimate is below the illustrative comparison value, but its upper bound crosses it, so the complete packet routes to review. Capture, scan, and evaluation timestamps must all be finite before freshness or pairing can support that result. The incomplete packet first requests the missing rear or side evidence. Neither accepted state means “safe to move.” The fixture assumes a supplied maximum overhang measurement; production measurement must establish which surfaces and directions that maximum actually covers.

Model Coverage at the Load Level

A load record should list required sides, visible footprint, top coverage where relevant, and regions hidden by wrap or neighboring goods. Declared side names are only a toy proxy. A real validator needs measurement-quality criteria and masks, especially where clear film produces ambiguous edges. Unobserved regions remain unknown rather than being filled by a neat geometric box.

Repeated video frames are correlated and must not inflate the number of verified loads. Deduplicate observations by load association and session, while keeping independent viewpoints for surface coverage. If a load exits and returns after repacking, create a new load-state revision. Old measurements should not automatically describe the newly arranged cartons.

The broader warehouse picking workflow owns inventory and task authority. The retail shelf audit exercise develops the related empty-versus-unseen distinction. A conventional measurement pipeline and validator are usually sufficient; a specialist agent is justified only for genuinely different evidence or qualified review support.

Protect the Operational Boundary

Server-side authorization must bind warehouse, tenant, and role before raw media or load records are accessed. Treat barcode payloads, labels, and OCR text as untrusted data. A label saying “approved” does not grant movement authority. Avoid worker identity or productivity judgments from incidental video and minimize unnecessary personal imagery in review packets.

The snippet omits authentication, trusted load-state storage, real calibration, persistent approval, database transactions, and task-provider semantics. Human-approved verification tickets should use load revision, session, measurement revision, and policy version as their idempotency scope. After a timeout, reconcile authoritative task state before retrying. An image-processing completion event is not a handling authorization receipt.

Acceptance Tests and Evaluation

• Hide the rear carton while showing three sides; require incomplete coverage rather than a complete load envelope.
• Mix centimeter and millimeter records; reject the comparison until explicit conversion is validated.
• Associate a label from the previous pallet or supply nonfinite scan/capture times; require pairing or provenance abstention. A nonfinite evaluation time also blocks freshness.
• Supply a near-boundary overhang with substantial uncertainty; route the packet to review instead of comparing central values alone.
• Repack the load after capture; invalidate earlier state-specific measurements and ticket drafts.
• Retry an approved verification task after a lost response; reconcile one task without triggering equipment actions.

Compare against a baseline that forwards candidate bounding boxes without evidence gates. Measure wrong-load associations divided by all adjudicated associations, incomplete loads labeled complete divided by all incomplete loads, and review-worthy overhang cases retained divided by all independently adjudicated review-worthy cases. Include abstentions and missed captures in session totals rather than evaluating only successful detections.

Begin in shadow mode at one stationary capture station and keep every packet human-reviewed. Split evaluation by load and session, not neighboring frames. Track reviewer correction time and recapture burden alongside errors; these are proposed measurements, not deployment results. AI systems engineering should first prove the identity and unit contract before connecting any downstream workflow.

Sources

• W3C PROV overview
• OpenCV camera calibration tutorial
• NIST combined standard uncertainty