AWS 420: Deliver the final platform from repository to monitored production-shaped environment
Capstone outcome
Deliver a fictional order API from reviewed source to a monitored production-shaped environment. The result must be reproducible, least privilege, policy-gated, immutable, progressively deployed, observable, recoverable and auditable. A running endpoint without evidence does not pass.
Defined platform
The reference design uses a multi-account pipeline, ECR image, CloudFormation/CDK infrastructure, ALB and ECS service across two Availability Zones, SQS worker, encrypted data store, Secrets Manager, CloudWatch/OpenTelemetry telemetry, EventBridge/SNS response and protected CloudTrail/Config evidence. The supplied path uses plans and simulated execution histories; an optional sandbox may substitute smaller compatible resources.
| Plane | Required proof |
|---|---|
| Source/build | Review, immutable revision, deterministic dependencies, tests |
| Artifact | Digest, SBOM, provenance, scan, signature, encryption/retention |
| Identity | Federated sessions, separated roles, trust and PassRole denials |
| Infrastructure | Reviewed change set, policies, drift/rollback boundaries |
| Deployment | Same digest, progressive traffic, alarms and bake |
| Operations | User SLI, logs/traces, alert route, runbook and owner |
| Recovery/audit | Backup/restore, rollback, CloudTrail/Config and cleanup |
Delivery phases
Phase 1 produces requirements: user journeys, SLO, RTO/RPO, data classification, Regions/AZs, scale, budget, ownership, compliance and threat model. Write ADRs for compute, data, messaging, deployment and recovery. Reject at least one viable alternative per decision.
Phase 2 establishes repository controls and roles. Build from a pinned source revision in an isolated runner using temporary credentials. Run unit/integration, dependency, secret and IaC tests. Generate SBOM/provenance; push once; scan and sign the exact digest. Fail when coverage/evidence is missing.
Phase 3 synthesizes and reviews IaC. Analyze replacement/deletion, IAM/KMS/network and quota/capacity effects. Execute only under an environment deployment role that can pass the exact stack execution role. Verify Config/security enrollment, logging and tags after deployment rather than trusting stack success.
Phase 4 deploys the approved digest to a canary or blue/green target. Run readiness, functional, negative authorization, data correctness, load and failure tests. Shift traffic in measured steps with user-centered alarms and a bake period. The approver sees immutable evidence and is independent of source/build/deploy authority.
Phase 5 proves operations: dashboards for availability/latency/correctness/queue freshness, bounded-cardinality logs/traces, alarm ownership, EventBridge routing with DLQ, runbooks, Health events, backup/restore, automated remediation constraints and incident communications. Record steady-state cost and headroom.
Operational handoff gate
Before declaring the platform delivered, run an operational readiness review. Confirm named service/business/security owners, on-call and escalation, SLO/error budget, quotas and scaling math, dependency contracts, data classification/retention, certificate/secret/key renewal, patch and vulnerability ownership, backup restore, regional/AZ behavior, deployment freeze/rollback, customer communications, cost anomaly response and decommissioning. Every item links to evidence, owner and blocking or expiring exception.
Have a second operator execute the release, alarm and restore runbooks from a clean session. Record ambiguities and repair the documentation or automation. A runbook written by the builder but never followed independently is not accepted. Export the final desired-state, artifact and evidence indexes so recovery does not depend on the original pipeline console remaining available.
Failure and recovery proofs
Inject or analyze ten failures: build dependency unavailable, artifact KMS denial, scan database unavailable, CloudFormation replacement risk, task fails readiness, one AZ unavailable, queue backlog, secret rotation with stale cache, alarm route target throttled and rollback after data schema change. For each predict, observe, contain, recover and verify user/data/security outcomes.
Use a release ledger binding source commit, pipeline/execution, artifact digest, signing/scan IDs, IaC template/change set, approval, task definition/runtime digest, traffic step, SLI/alarm state, CloudTrail request IDs and cleanup. Hash supplied artifacts so another learner can reproduce the result.
~~~bash aws sts get-caller-identity aws codepipeline get-pipeline-execution --pipeline-name PIPELINE --pipeline-execution-id ID aws ecr describe-images --repository-name app --image-ids imageDigest=sha256:DIGEST aws cloudformation describe-change-set --stack-name APP --change-set-name RELEASE aws ecs describe-services --cluster PLATFORM --services order-api aws cloudwatch describe-alarms --alarm-name-prefix OrderApi ~~~
Security, cost and cleanup
Prove denied paths: build cannot deploy/sign production, deploy cannot edit IAM/pipeline/artifact, runtime cannot read unrelated secrets, application subnets lack unintended ingress/egress and evidence administrators cannot silently mutate originals. Resolve every exception or record owner/expiry/compensation.
Estimate monthly normal and degraded capacity, logs/traces/custom metrics, scans, Config/CloudTrail data events, queues, NAT/data transfer, backups and retained artifacts. Optional execution requires a budget alarm and named cleanup manifest. Delete application resources in dependency order, retain only policy-required evidence, verify no ENIs/load balancers/endpoints/snapshots/secrets/keys/logs remain, and perform a second inventory pass.
Failure matrix
Diagnose 24 hazards: branch unprotected, mutable dependency, cache untrusted, static key, shared role, PassRole broad, digest absent, SBOM detached, scan failed clean, signer role broad, cross-account KMS denied, change set skipped, destructive replacement missed, Config recorder absent, canary capacity wrong, alarm missing data, SLI component-only, trace context broken, EventBridge duplicate action, DLQ ignored, secret logged, restore untested, rollback data-incompatible, and cleanup incomplete.
Acceptance
Submit requirements/threat model, five ADRs, source/pipeline/IaC, role and policy tests, artifact evidence, change set, progressive deployment, telemetry/response, ten failure proofs, restore/rollback, cost and cleanup ledger. Pass requires every plane in the table, same-digest promotion, customer-level verification, denied-path evidence and independent reproducibility.