Agent Release-Gate Kit

1. Evaluation release gate

A high average can hide the one failure you cannot ship. The gate decides on every case, not on the mean.

• Blocks a release when any critical case fails, even at 99% overall
• Treats an ungraded case as a failure to grade, never as a pass
• Rejects a run with missing or duplicated cases, so a shrinking test set cannot pass quietly
• Applies your quality threshold separately from the critical-case rule

2. Retry-safe tool actions

Agents retry. A tool that refunds, emails or writes must do it once, however many times it is called.

• Records each change with its operation key in the same transaction as the change
• Replays the stored result when a response was lost after the commit
• Commits one change when two retries arrive at the same moment
• Refuses an operation key reused with a different payload

3. Tenant isolation

The model should never decide whose data it is working on. The tool layer decides.

• Blocks reading, changing or searching another tenant’s records
• Refuses a tool call that tries to name the tenant in its arguments
• Gives the same answer for another tenant’s record as for a missing one
• Still lets the agent do its own tenant’s work, so a layer that refuses everything cannot pass

4. Before launch: a short check

Each line maps to a section of the AI Launch Readiness Checklist.

• The release gate runs on every change, and critical cases block on a single failure
• Every case in the required set is graded; ungraded never counts as a pass
• Every tool that changes data or sends something is safe to retry
• You know which actions a rollback cannot undo, and how to reconcile them
• Tenant and permission checks live in the tool layer, not in the prompt
• A foreign record and a missing record look the same to the agent
• A named owner reads the gate results before each release

Run every gate with one command

The runner checks each gate’s tests and compares its output with the results recorded when the kit was packaged. It exits with an error if anything fails, so the same command is your CI step.

unzip agent-release-gate-kit-v1.zip
cd agent-release-gate-kit
python3 -B run_all.py