AI Launch Readiness Checklist
The questions behind the Architecture & Harness Review, used to decide whether an AI workflow or agent is ready for business use. Work top to bottom — anything you can’t check is a risk worth scoping before launch, not after.
This checklist covers runtime behavior and launch ownership. Its companion covers the process that produced the code.
Inspect how the software was built1. Use-case fit
Before anything is built, confirm the workflow is worth automating and the AI is allowed to do the job.
- The specific decision or task the AI system owns is written down in one sentence
- It is clear whether the system is internal, customer-facing, or revenue-generating
- The cost of a wrong output, leaked data, or non-adoption is quantified
- A measurable definition of “reliable enough for this business” exists
- There is a deadline or event forcing action, and a named owner of the outcome
2. Architecture & integration
The model is the easy part. The system around it is where live breaks.
- Workflow is mapped end to end, including the human steps the AI replaces or assists
- Tool and system boundaries are explicit — what the AI can read vs. write
- Integration points with your business systems have defined contracts and failure handling
- Latency and cost budgets are set per step, not just for the whole call
- The architecture degrades safely when the model is slow, wrong, or unavailable
3. Data & permissions
Permission and data-boundary failures can expose information or allow actions beyond the caller’s authority. Test these boundaries explicitly.
- Every data source the system can touch is inventoried and access-scoped
- Permissions are enforced at the tool layer, not assumed in the prompt
- PII, secrets, and regulated data have explicit handling and retention rules
- Retrieval sources are versioned and their freshness is monitored
- There is a clear boundary between what the AI can see and what it can act on
4. Evals & correctness
If you cannot measure correctness before and after launch, you cannot ship responsibly.
- A representative eval set exists and is run on every meaningful change
- Correctness is measured against business outcomes, not just model metrics
- Regression detection runs automatically before a release ships
- Edge cases, adversarial inputs, and known failure modes are in the eval set
- Mandatory human-review points are defined for high-stakes decisions
5. Observability & rollback
You need to know when it breaks, why, and how to stop it — fast.
- Traces capture the necessary inputs, tool calls, and outcomes, with sensitive data redacted and retention controlled
- Cost, latency, and error rates are monitored with alerting thresholds
- Failures are detected and escalated through a defined incident path
- Recovery has been tested, including effects a rollback cannot undo and actions that need reconciliation
- There is a kill switch to disable the system without a full deploy
6. Ownership & launch
A system nobody owns after go-live is a system that quietly decays.
- Launch gates are defined and signed off before any rollout
- A named team owns the system after go-live, including evals and incidents
- A support model exists for when the AI gets it wrong in front of a user
- Rollout is staged (internal → limited → general), not all-at-once
- Success and retirement criteria are agreed before launch, not after
Want this run against your actual system?
The Architecture & Harness Review runs these questions against your system and ends in a prioritized plan. If the scope is already clear, we can scope a forward-deployed build directly.
Explore the review