A reference architecture with six responsibilities
A framework may package several responsibilities together, but they still need owners and contracts. Start with the product’s task and data boundaries. Then decide where runtime judgment belongs. Orchestration patterns vary with the work; no single arrangement is appropriate for every agent.[1]
| Boundary | Contract | Evidence to inspect |
|---|---|---|
| Entry | Authenticate the caller; derive tenant and task context on the server. | Denied requests and cross-tenant tests. |
| Task state | Record a task ID, version, lifecycle, budget and terminal outcome. | A restart resumes or reconciles a task without inventing history. |
| Model runtime | Supply authorized context and allowed tools; bound iterations and cost. | Model and prompt versions, stop reasons and route decisions. |
| Policy and tools | Authorize the current caller for the resource and proposed action. | Tool schemas, policy decisions and rejected arguments. |
| Business execution | Apply validated mutations through the domain contract. | Idempotency, concurrency checks and an auditable result. |
| Evidence and operation | Connect task outcomes to traces, evaluation cases and recovery. | A failed task can be investigated without broad sensitive-data access. |
Walk through a support-credit request
In this synthetic example, a support operator asks the product to investigate a service interruption and propose an account credit. The system derives the operator’s tenant and permissions from authenticated context. It creates a task record before investigation, then offers read tools for that account’s service history and billing policy.
The model retrieves relevant evidence and proposes a credit amount. The tool boundary checks whether the records are accessible and whether the operator may propose that kind of change. The proposal includes the account version and source references. A human approval step, where required by policy, is bound to this exact proposal.
At commit, the domain service rechecks authority, validates the current account version and applies the mutation under a stable operation ID. The task records the committed result. If the response is lost after the write, the next step is to look up that operation; starting a new credit request would be an incorrect recovery.
Keep conversation, task state and business state distinct
Conversation history is context for an interaction. Task state says what the execution has attempted and what remains unresolved. Business state is the authoritative account, order or document record. Treating the chat transcript as all three makes recovery and access control difficult.
- Task state: proposed, awaiting approval, executing, completed, rejected, failed or outcome unknown. Choose names that match the domain.
- Business state: written only through the system of record, using its existing validation and concurrency rules.
- Context: retrieved or remembered information with scope, provenance and freshness rules. A previous observation is not proof it is still true.
- Evidence: operational events and selected artifacts with retention and access rules, rather than an unrestricted transcript archive.
A runtime restart must not turn an executing task into a fresh task automatically. Reconcile the business operation first. A task ID and a model conversation ID can be related without being interchangeable.
Place authority where the action is enforced
Authorization should be checked for the requested resource and operation on every relevant request. Tool discovery, a valid login and a prompt instruction do not replace that decision. OWASP’s authorization guidance emphasizes server-side checks and default denial; applying those principles to agent tools is an application design responsibility.[2]
MCP can expose tools and other capabilities to clients. It does not remove the underlying domain API, database or policy boundary. Keep protocol authentication, user delegation and resource authorization explicit. This guide uses the 2026-07-28 MCP specification as its protocol reference.[3]
Specify trusted identity, resource scope, allowed action and commit-time checks.
Review the architecture through its failure paths
| Trigger | Expected behavior |
|---|---|
| Permission revoked during investigation | The next read or write is denied according to current policy; stale context does not restore access. |
| Model provider unavailable | A tested, policy-compatible fallback is used or the task stops with a clear state. |
| Tool write times out | The task records an unknown outcome and reconciles the original operation ID. |
| Approval arrives after a record changes | Version mismatch blocks the stale proposal and requests a new decision. |
| Task exceeds budget | The loop terminates with its evidence and an explicit unresolved outcome. |
| Worker restarts | Durable state determines whether to resume, reconcile or stop. |
Ask the engineering partner to demonstrate these paths against a small fixture. A diagram that cannot explain them is incomplete, even if its boxes have familiar technology names. For external side effects, identify which operations are reversible, compensatable or require manual recovery.
What a CTO should require from the design
The design package should include a task lifecycle, domain/API contracts, identity and data-flow diagram, failure-state handling, evaluation approach and operating responsibilities. It should name the source of truth for each decision. A prototype may keep several components in one service; clear boundaries do not require premature microservices.
Use this architecture as a review framework, then simplify it to the actual product. A read-only research assistant and an agent that changes billing records have different consequence levels. The right design makes those differences visible in its controls, experiments and release criteria.
Questions this guide answers
What are the essential boundaries in an AI agent architecture?
Define identity, durable task state, authorized tools, business mutations and execution evidence. A model may propose a step, but the system must validate whether that step is permitted and still applicable. The design should explain completion and failure paths, including a lost response, revoked access or rejected proposal.
Where should authorization happen for an agent tool call?
At the trusted service boundary that can enforce the resource and operation policy. Check the authenticated caller, tenant, target resource and requested action using current state. A permission described in a prompt or checked only when the conversation began cannot protect a later mutation after access or business state changes.
How should the system recover when a tool response is lost?
Preserve the operation identity and determine whether the original action committed before issuing another write. The recovery contract should distinguish pending, completed, rejected and uncertain outcomes. Idempotency and reconciliation need to cover the actual business effect; repeating a model call alone does not establish that a retry is safe.
Sources and scope
Technical references inform the cited statements. The decision frameworks and synthetic examples are TeqEngine’s editorial guidance.
- Microsoft: AI agent orchestration patternshttps://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns
- OWASP: Authorization Cheat Sheethttps://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html
- Model Context Protocol: specification, 2026-07-28https://modelcontextprotocol.io/specification/2026-07-28