Agent failure labs / Guide 60
AI Deployment Approval: Yesterday's Yes Cannot Authorize Today's Build
Reproduce an AI production deployment approval bypass using a stale approval, then bind environment, artifact digest, reviewer, and expiry before release execution.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
“Deploy the approved checkout fix to production.” In this hypothetical request, the assistant finds a green approval record from yesterday. That approval covered a staging artifact. A rebuild has since produced a different digest, and the current request targets production. The assistant sees approved equals true and proceeds, converting an old decision about one artifact into permission for another release.
This lab isolates approval substitution. It is not a general incident-response architecture or a code-review tutorial. Passing tests, finding an approval message, and having deployment credentials are different facts. The release boundary must establish exactly which artifact, environment, configuration, and operation a qualified reviewer approved before anything can change in production.
Reproduce the Reusable-Boolean Failure
The unsafe mock checks only an approved flag. The guarded function compares release identity, environment, digest, and configuration revision, then checks reviewer authority, separation from the requester, expiry, and test evidence. The result is a validation state, not a deployment. The fixture uses a fixed clock so repeated runs are deterministic.
This is a deterministic synthetic simulation of a possible failure, not an experiment against an actual LLM, CI provider, or cluster. Both functions receive the same stale approval and release proposal. No network request, deployment command, secret lookup, or filesystem mutation occurs. Artifact strings and reviewer identifiers are fictional values, not cryptographic attestations.
proposal = {"operation": "release-9", "environment": "production",
"digest": "sha256:new", "config_revision": 12,
"requester": "builder", "tests_passed": True}
stale = {"approved": True, "operation": "release-8", "environment": "staging",
"digest": "sha256:old", "config_revision": 11,
"reviewer": "release-owner", "expires": 200}
authorized_reviewers = {"release-owner"}
now = 100
def naive(release, approval):
return "mock_deploy" if approval["approved"] else "hold"
def guarded(release, approval, clock):
if approval.get("approved") is not True:
return "hold"
if any(release[k] != approval.get(k) for k in ("operation", "environment", "digest", "config_revision")):
return "approval_mismatch"
if approval.get("reviewer") not in authorized_reviewers or approval["reviewer"] == release["requester"]:
return "reviewer_denied"
if clock >= approval["expires"]:
return "expired"
return "ready_for_deployment_job" if release["tests_passed"] is True else "tests_missing"
assert naive(proposal, stale) == "mock_deploy"
assert guarded(proposal, stale, now) == "approval_mismatch"
fresh = dict(stale, **{k: proposal[k] for k in ("operation", "environment", "digest", "config_revision")})
assert guarded(proposal, fresh, now) == "ready_for_deployment_job"
assert guarded(proposal, fresh, 200) == "expired"
assert guarded(proposal, dict(fresh, reviewer="builder"), now) == "reviewer_denied"
assert guarded(dict(proposal, digest="sha256:rebuilt"), fresh, now) == "approval_mismatch"
assert guarded(dict(proposal, tests_passed=False), fresh, now) == "tests_missing"
print("Stale approval blocked; matching release can reach its restricted job.")The fixed rerun rejects yesterday’s approval on exactly the input accepted by the unsafe path. A matching approval then reaches a restricted-job state. Rebuilding the artifact breaks that binding again. The example’s tests_passed Boolean is deliberately a trusted fixture input; production must derive it from verified CI evidence tied to the same artifact, not from the agent’s claim.
Approval Must Describe the Release
“Only deploy approved changes” is a useful prompt but not an authorization mechanism. Approval may refer to a pull request, staging exercise, previous commit, or emergency rollback. A retrieved chat message can also be forged or quoted out of context. The agent should identify candidate evidence and summarize it, while the release service verifies origin and exact operation matching.
OWASP transaction authorization guidance recommends acknowledging significant data, preventing modification after approval, and checking authorization at execution. For a release, significant data includes the immutable artifact digest, target environment, configuration revision, migration plan, blast radius, and recovery reference. A branch name or mutable image tag is not an adequate artifact identity.
The proposal should display those fields to the reviewer along with test provenance and relevant differences from the currently running release. If a migration or configuration changes after review, create a new proposal. Do not let the model decide that the change is too small to need renewed approval. The policy owner, not generated commentary, defines any preauthorized exception class.
Enforce the Boundary in the Delivery Platform
GitHub’s environment documentation explains that jobs referencing an environment must satisfy its protection rules before running or accessing environment secrets. It also documents reviewer controls, prevention of self-review, and configurable administrator bypass behavior. Verify availability for the repository plan and visibility rather than assuming every protection is universally enabled.
The same documentation notes that referencing a nonexistent environment can create one without configured protection rules. Therefore, a model must not be able to choose arbitrary environment names or edit the workflow enforcing its own gate. Preconfigure approved environments, restrict workflow changes, and ensure deployment credentials cannot be reached through an alternate unprotected job.
OWASP authorization guidance recommends least privilege and permission checks on each request. Separate the assistant’s read access to CI evidence from permission to request a release and from the deployment job’s runtime credentials. Reviewer identity should come from the trusted identity and approval service, with current entitlements checked at execution.
One release orchestrator and deterministic checks are enough for the first implementation. A specialist agent may summarize security scan findings or migration risk, but it should not approve its own output or alter credentials. Model consensus cannot replace a named release owner, an authenticated approval record, or a provider-enforced environment boundary.
Reconcile Running State, Not Just Job Status
Persist a stable release operation before dispatching the job. If dispatch or execution times out, record an unknown outcome and inspect the authoritative workflow run and runtime deployment state. Starting another job under a new identifier can produce overlapping rollouts. Test the provider’s actual retry, cancellation, and concurrency semantics before permitting automated recovery.
A returned status: success from a wrapper is not a trustworthy receipt without authenticated origin and a match to the release operation, environment, and artifact. Even a successful CI job does not prove every instance is running the intended digest or that health checks cover the business-critical path. Reconcile observed runtime state and keep partial rollout distinct from complete deployment.
Rollback is another bounded operation, not a universal undo button. The previous application artifact may be incompatible with a forward-only database migration. Require a reviewed recovery plan and explicit authorization for its scope. If runtime health is uncertain, stop expansion and page the owner rather than asking the model to invent a repair command.
The snippet omits the authentication source, database transactions, persistent approvals, state persistence, provider semantics, signed artifacts, real test attestations, and all side effects. It assumes trusted evidence and a single immutable proposal. Deterministic field comparisons cannot guarantee a safe release when tests are inadequate, credentials bypass the gate, or artifact provenance is compromised.
Acceptance Tests
- A staging approval for an older digest must not authorize the current production proposal.
- A changed artifact, configuration revision, migration plan, or operation identifier must invalidate earlier approval.
- An expired approval or a reviewer without current production authority must stop the deployment job.
- A requester attempting self-approval must fail even when they can edit the assistant’s conversation.
- An unknown environment name or alternate unprotected workflow must not provide a path to production credentials.
- A dispatch timeout or partial rollout must reconcile the original operation before any retry or rollback is authorized.
Roll Out with Bypass Tests
Establish a baseline of release proposals labeled with the exact approval they require. Include stale digests, renamed environments, revoked reviewers, and altered migration plans. Measure unauthorized proposals reaching a deployment job over all deliberately unauthorized attempts. Separately measure valid releases blocked over all approved matching proposals, and report reviewer delay and reconciliation age.
Start by validating historical proposals without dispatching jobs, then shadow the current release process. Pilot a narrow human-approved service with a bounded rollout and an independent stop mechanism. A proposed guardrail is no credential access before a matching approval, with suspension after any bypass. These are testable targets, not evidence of a deployed system’s reliability.
The GitHub code review guide covers upstream review evidence; the database update safety lab shows why version-bound approval matters during mutation. DevOps and platform engineering can establish the enforced delivery boundary. Start with the stale-approval fixture and an attempted alternate credential path, not a more confident release summary.