10. Architecture Decisions and Production Readiness

10.1 Preserve why the system is shaped this way

An architecture diagram records what exists. It rarely records why one design was chosen, which alternatives were rejected, what evidence supported the choice, or what would cause the team to reconsider it. Those details disappear into meetings and chat, then return months later as repeated debates or unsafe changes.

An Architecture Decision Record (ADR) captures one consequential decision and its context, alternatives, trade-offs, and consequences. The collection forms a chronological decision log. The practice is intentionally lightweight; the current UK Government ADR framework, published in 2025, likewise focuses on establishing consistent documentation of architectural decisions during system design and development.¹

Write an ADR when a decision materially affects security, reliability, data handling, behavioral quality, cost, operability, organizational ownership, or future reversibility. Typical AI-platform triggers include:

Direct provider access versus a shared AI gateway
Application-owned harness versus a common agent runtime
Retrieval-time authorization and source-of-truth choices
Model, embedding, reranking, or hosting strategy
Tool permissions, approval boundaries, and sandbox isolation
Memory scope and retention
Evaluation gates and acceptable residual risk
Regional processing, recovery architecture, and provider fallback
Build-versus-buy choices that create data or operational dependency

Do not create ADRs for every parameter adjustment or implementation detail. Use standards and reference architectures for repeated decisions; create an ADR when a team adopts, varies from, or replaces them.

Google’s ADR guidance defines the same core purpose: capture significant requirements, options, decisions, and the history needed to understand later architectural evolution.² Store ADRs close to the versioned system, make them searchable from the service catalog, and link them to code, release bundles, threat models, evaluations, incidents, and runbooks.

10.2 Use a small, AI-aware ADR format

Field	What to record
Identifier and title	Stable number plus a decision statement, such as “Use permission-filtered hybrid retrieval”
Status	Proposed, accepted, rejected, deprecated, or superseded; link the replacing ADR
Date, owner, reviewers	Who is accountable and which disciplines reviewed it
Context and scope	The problem, affected users, system boundary, constraints, and deadline
Decision drivers	Quality, security, latency, cost, residency, team capability, reversibility, and other ranked needs
Options considered	Serious alternatives, including “do nothing,” with evidence-based advantages and disadvantages
Decision	The chosen option, exact boundary, and important configuration assumptions
Consequences	What becomes easier, harder, more expensive, or newly risky
Validation	Evaluations, prototypes, load tests, threat models, and operational exercises supporting the choice
Guardrails and fitness functions	Automated checks that detect erosion of the decision’s assumptions
Reversal and review triggers	Migration path, switching cost, review date, and conditions that reopen the decision

Table 10-1. Context, decision, and consequences are the irreducible core. AI systems also need behavioral evidence and explicit review triggers because models, providers, and risks evolve quickly.

stateDiagram-v2
    [*] --> Proposed
    Proposed --> Accepted: decision approved
    Proposed --> Rejected: alternative declined
    Accepted --> Deprecated: still present, no longer preferred
    Accepted --> Superseded: replacement ADR accepted
    Deprecated --> Superseded: replacement completed
    Rejected --> [*]
    Superseded --> [*]

Figure 10-1. ADR status drives lifecycle behavior; it is more than document decoration. A changed decision receives a successor so the evidence available at each point remains legible.

An ADR is not a sales comparison or a retroactive justification. State uncertainty, evidence gaps, dissent, and negative consequences plainly. Prefer measured workload results over generic benchmark rankings. Record pricing assumptions with a date, but make the decision depend on workload economics rather than a price that will soon change.

Treat accepted ADRs as append-only history. Correct spelling or broken links in place; supersede a changed decision with a new record rather than rewriting why the old team acted. I prefer short ADRs with explicit reversal triggers to polished essays that conceal uncertainty. The record exists to make the next decision better.

10.3 Example: fail closed when policy evidence is unhealthy

The following condensed ADR shows the required level of specificity.

ADR-014 — Require current, permission-verified evidence for policy answers
Status: Accepted
Context: The Operations Copilot answers company-policy questions. The model’s pretrained knowledge may be outdated, and the retrieval system can become stale or unavailable. A plausible but unsupported answer may influence employee decisions.
Decision drivers: Currentness, tenant isolation, explainability, predictable degraded behavior, and recovery time.
Options: (A) answer from model knowledge when retrieval fails; (B) answer with a warning; (C) withhold the policy answer and provide approved source links or human escalation.
Decision: Choose C. Material policy claims require permitted evidence from an index within the freshness objective and citations that survive validation. Otherwise the product reports that verified policy information is unavailable.
Consequences: Some requests fail during knowledge outages, reducing apparent availability. The system needs source-health telemetry, a source-link fallback, clear user language, and support escalation. It prevents ungrounded model memory from silently becoming company policy.
Validation: Retrieval ACL suite, stale-index simulation, citation checks, degraded-mode game day, and user-comprehension test.
Fitness functions: Block release if an unauthorized passage reaches context or a material claim lacks evidence; alert when freshness exceeds its objective.
Review triggers: A verified authoritative policy API becomes available, the risk classification changes, or observed abstention materially harms the business process.

The decision is useful because an engineer can implement and test it, an operator knows the degraded mode, and a future team knows what evidence could justify a different choice.

10.4 Make readiness an evidence decision

A production-readiness review asks whether the assembled system is fit to serve a defined population under declared conditions. It is not a demonstration, a checklist signed from memory, or a last-minute security meeting. Start the review while architecture is still changeable, then update evidence as the release matures.

flowchart LR
    U["Use case, users, risk tier"] --> A["Architecture and ADRs"]
    A --> I["Implementation and release bundle"]
    I --> E["Evaluation, threat, load and recovery evidence"]
    E --> R{"Readiness decision"}
    R -->|"ready"| L["Controlled launch"]
    R -->|"conditional"| C["Limited scope + expiring conditions"]
    R -->|"not ready"| F["Remediate or stop"]
    L --> M["Production monitoring"]
    C --> M
    M --> N["New evidence, incidents, changed assumptions"]
    N --> A

Figure 10-2. Readiness is a lifecycle checkpoint within continuing governance. Production evidence can reopen an ADR or withdraw approval.

The review packet identifies the exact release bundle, environment, user population, enabled capabilities, data classes, risk tier, and exposure plan. Evidence from a different model route, prompt, index, or tool contract does not approve the candidate.

AWS’s 2025 Responsible AI Lens similarly recommends defining testable release criteria and making an evidence-based release decision across the full application lifecycle.³ A readiness process should reuse existing engineering evidence rather than ask teams to transcribe it into a separate governance system.

10.5 Production-readiness checklist

The following checklist is a minimum platform standard. Riskier use cases add domain, legal, compliance, accessibility, fairness, or independent-assurance requirements.

Purpose and accountability

The user problem, intended population, business outcome, and non-goals are documented.
The team demonstrated why AI is appropriate and identified a deterministic or human fallback.
Product, engineering, operations, data, security, and risk owners are named.
The risk tier, prohibited uses, affected stakeholders, and human decision authority are explicit.
Reviewers and approvers have defined decision rights, usable evidence, workload limits, training, calibration, and escalation paths.
User disclosures, escalation, feedback, and contestability paths are ready.

Architecture and dependencies

The end-to-end architecture, trust boundaries, data flows, external effects, and failure domains are current.
Consequential design choices have accepted ADRs with assumptions and review triggers.
Every model, provider, library, tool, connector, dataset, index, and policy artifact has an owner and version.
Provider quotas, regional availability, deprecations, contracts, and correlated dependencies are understood.
No prototype-only component or unmanaged production-console configuration remains.

Identity, data, security, and safety

Identity and authorization are enforced before retrieval and at every tool boundary.
Tenant isolation, least privilege, delegated credentials, approvals, and revocation have been tested.
Data classification, provenance, residency, retention, deletion, logging, and provider use are approved.
Threat modeling covers prompt injection, excessive agency, disclosure, poisoned content and memory, supply chain, and resource abuse.
Model outputs are typed and validated; code execution, files, credentials, and network egress are deterministically contained.
Red-team and security tests include indirect attacks through retrieved content and tool output.

Behavioral evidence

Success, unacceptable behavior, and hard gates are defined before launch.
Versioned evaluations cover retrieval, generation, tools, trajectories, outcomes, safety, latency, and cost.
Results meet gates overall and on important language, identity, geography, tenant, and tool-path slices.
Model graders are calibrated; critical claims use deterministic or expert verification where possible.
Shadow or replay results and candidate-versus-production comparisons are reviewed.
Known limitations and residual risks have owners and are reflected in the product experience.

Operations, reliability, and recovery

User-level SLIs, SLOs, error budgets, dashboards, alerts, and runbooks are operational.
Traces correlate release, identity, sources, models, policy, tools, cost, and verified external effects without over-logging sensitive data.
Operators have practiced distinguishing behavioral, retrieval, policy, tool, dependency, and telemetry failures and selecting the corresponding safe degraded mode.
Timeouts, retries, idempotency, reconciliation, circuit breakers, concurrency limits, and graceful degradation have been exercised.
Capacity and provider quotas have been load-tested at expected peak plus declared headroom.
RTO, RPO, recovery state, last-known-good artifacts, and external-system reconciliation are documented.
Restore, regional/provider failover, control-plane loss, and rollback have passed recent exercises.

Delivery, economics, and support

The immutable release bundle pins code, model route, prompts, index, tools, policies, graders, and runtime limits.
CI/CD gates, separation of environments, canary plan, stop criteria, rollback, and migration compatibility are tested.
Feature flags and temporary exceptions have owners and expiry dates.
Fully loaded cost per valid outcome, team allocation, budget controls, and growth forecast are acceptable.
Support coverage, incident severity, communications, vendor escalation, and business-continuity processes are staffed.
Post-launch review dates, evidence refresh, model/index migration, and eventual decommissioning are planned.

This is deliberately broader than a model card or conventional service review. AWS’s 2025 production framework also emphasizes measurable business impact, validation under real constraints, and operability rather than treating a compelling demonstration as production proof.⁴

10.6 Record the decision, not only the meeting

The readiness record should have one of three outcomes:

Outcome	Meaning	Required record
Ready	Evidence supports the defined launch scope	Release, scope, approvers, evidence links, rollout and review date
Conditionally ready	A bounded gap is accepted for limited exposure	All above plus risk owner, compensating control, deadline and automatic expiry
Not ready	A hard gate failed or evidence is insufficient	Failed criteria, owner, remediation or stop decision

Table 10-2. “Conditional” is not a softer synonym for ready. Some failures—such as cross-tenant disclosure or unapproved consequential action—are hard stops.

Exceptions must be specific: affected capability, population, environment, residual risk, compensating control, approver, expiry, and monitoring. They should expire automatically rather than becoming permanent architecture. A limited launch reduces exposure only when limits are technically enforced.

10.7 Assemble the Operations Copilot readiness packet

For the first production launch, the packet would include:

Release manifest for the exact model route, prompts, policy index, case tool, policy bundle, and evaluators
ADR-014 plus decisions for gateway use, delegated service-desk identity, approval, index migration, and recovery region
Permission and retrieval tests; grounded-answer, abstention, injection, and case-effect evaluations; load and cost reports
Data-flow and threat models, provider review, telemetry and audit design, and red-team findings
SLOs, dashboards, alerts, runbooks, degraded modes, case reconciliation, rollback, and a completed recovery exercise
Known limitations, employee guidance, escalation route, named owners, rollout cohort, stop criteria, residual-risk acceptance, and 30-day review

The review should reject the launch if, for example, the evaluation used an older index, service-desk writes were never tested against ambiguous timeouts, the recovery region lacks the approved model, or the owner cannot query which employees received a faulty answer. Those are not documentation gaps; they are missing production capabilities.

ADRs and readiness reviews close the handbook’s engineering loop. ADRs preserve the reasoning behind durable choices; readiness reviews test whether those choices have become an operable, secure, evaluated, recoverable, and economically justified production system.