Lesson 076 · AWS Learning Path

AWS 076: Status checks and automatic recovery

· Published · 7 min read

System, instance, and attached EBS health signals feed different repair, automatic recovery, and replacement responses

The problem

A team uses Status checks and automatic recovery as a label or Console setting without proving the identity, scope, behavior, failure boundary, cost, or operational result. The configuration appears complete but the real requirement remains untested.

Final outcome

The learner will inspect, explain, test, and document Status checks and automatic recovery. The submission must connect the Console and CLI view to the same AWS control, state what the evidence proves, diagnose one likely failure, and make a requirement-based architecture decision.

Learning outcomes

You will be able to:

  1. explain ec2 runs three status-check categories;
  2. explain checks run every minute;
  3. explain system failure points below the guest;
  4. explain instance failure points inside the guest;
  5. explain automatic recovery preserves identity;
  1. distinguish successful configuration from successful workload behavior;
  2. preserve redacted evidence and complete the stated cleanup or retention decision.

Mental model

requirement
   -> identity and permission
   -> account, Region, and resource scope
   -> configuration or request
   -> observable state
   -> workload result
   -> failure evidence
   -> cleanup or controlled retention

Never begin with a create button or a copied command. Start with the result that must be proven and the boundary that must remain protected.

Core facts

ConceptWhat it means in practice
EC2 runs three status-check categoriesSystem status checks cover AWS host infrastructure, instance status checks cover guest reachability and configuration, and attached EBS status checks cover EBS reachability and I/O on supported Nitro instances.
Checks run every minuteEach returns pass or fail and publishes CloudWatch status metrics. Status checks cannot be disabled or deleted.
System failure points below the guestPower, host hardware, or physical-network problems can fail the system check. Stop/start, EC2 recovery, or AWS remediation can move or restore the instance.
Instance failure points inside the guestBad network or startup configuration, exhausted memory, corrupted file system, or kernel trouble can fail the instance check and normally needs customer action.
Automatic recovery preserves identitySupported recovery after system-check failure retains the instance ID, AZ, IP addresses, metadata, placement group, and EBS volumes, but RAM and operating-system uptime are lost.
Status checks are not application checksAn instance can pass EC2 checks while its process, dependency, TLS certificate, or business function is broken. Use load-balancer health checks, synthetic tests, logs, and application metrics.

How to reason about it

Recovery keeps one instance identity after eligible host failure; replacement creates a fresh instance from known configuration. For stateless services, Multi-AZ Auto Scaling replacement usually provides stronger application resilience. Use recovery when preserving a particular instance identity is a documented requirement, and still protect data independently.

Use the following decision table as a starting point, then change the answer when the scenario changes.

RequirementPreferred directionWhy
System status check fails on a supported singletonAutomatic recovery or stop/startThe likely problem is the underlying host.
Instance status check failsGuest troubleshooting, reboot, or replacementThe operating system or its configuration normally needs customer action.
Attached EBS status check remains impairedInvestigate volume and host path; fail over or replaceThe check identifies EBS reachability or I/O impact on supported Nitro instances.
Web process is unhealthy but EC2 checks passLoad-balancer health replacement and application diagnosticsInfrastructure reachability does not prove the service works.

Prerequisites, permissions, Region, and cost

Run this lesson as the approved non-root course identity. Begin with aws sts get-caller-identity, confirm the private account record, and set the fixed project Region before any regional query. IAM resources are global within an account, while STS endpoints and the services reached by an identity can be regional.

Use only the read or change actions required for this lesson. An AccessDenied result is evidence to analyse, not permission to switch to root or attach AdministratorAccess. Record the action, resource, principal type, request context, and smallest justified correction.

The listed cost tier is T0 - no resource creation. Free Tier eligibility and credits are account-specific. Before a mutating lab, identify every resource that can charge, estimate its duration, start a timer, and write the reverse cleanup order. A budget reports cost after billing data arrives and is not a real-time stop control.

AWS Management Console method

  1. Open EC2 Instances, select an instance, then Status checks and Monitoring.
  2. Compare System reachability, Instance reachability, and attached-EBS status with application and load-balancer health.
  3. Open CloudWatch Alarms and inspect the StatusCheckFailed metrics and recovery or notification action without changing production alarms.

For every step record the service page, selected account and Region, exact object, visible state, and why that state matters. A Console label or green status is not enough unless it is tied to the final workload outcome.

AWS CLI or API evidence

Read all status categories and instance events before choosing recovery.

aws ec2 describe-instance-status --instance-ids i-0123456789abcdef0 --include-all-instances --query 'InstanceStatuses[0].{State:InstanceState.Name,System:SystemStatus.Status,Instance:InstanceStatus.Status,EBS:AttachedEbsStatus.Status,Events:Events}'

Impaired System suggests underlying infrastructure. Impaired Instance suggests guest trouble. A missing EBS field can reflect instance support or no reported impairment; confirm metrics and documentation.

Before running the command, replace every placeholder, explain each option, and decide whether the operation is read-only or mutating. Capture the exit code immediately. Redact account IDs, ARNs, public addresses, request identifiers, and personal data before sharing.

The CLI and Console are clients of AWS APIs. Matching state across them increases confidence, but neither substitutes for data-plane or application verification.

Practical work

Start nw-p04-web-1 for a maximum 15-minute evidence window. Inspect system, instance, and attached-EBS status checks plus application, SSM Agent, and Apache health. Create p04-health-map.md that maps each failed signal to owner, likely layer, evidence, stop/start or recovery implications, CloudWatch alarm metric, and safe response. Inspect automatic recovery options without enabling a paid or unreviewed action, then stop the instance unless AWS 077 starts immediately.

The evidence package must include:

  • non-root principal type, account verified privately, and Region;
  • exact intended and observed state;
  • one Console observation and the matching CLI or API result;
  • one successful result and one denied, failed, or counterexample result;
  • what each result does not prove;
  • cost state and elapsed lab time;
  • cleanup evidence or a named retained-state owner and next lesson.

Verification standard

Use three levels of proof:

  1. Control plane: the object or policy exists with the intended configuration.
  2. Data plane or behavior: the request, packet, session, storage path, or application does what the requirement states.
  3. Operations: monitoring, failure diagnosis, cost, ownership, and cleanup are known.

A control-plane response can precede final readiness. Use waiters or state polling where supported, then test the actual behavior. If a request times out, do not assume it failed. Inspect state and use documented idempotency before retrying a mutation.

Troubleshooting method

SymptomEvidence firstSmallest safe response
command cannot authenticatecredential source, expiry, caller preflightrestore approved temporary login
access is deniedaction, resource, principal, all policy layerscorrect only the missing or conflicting control
object appears missingaccount, Region, filters, pagination, permissionalign scope before creating anything
configured state exists but behavior failsroute, identity, dependency, logs, service statetest the next boundary in the path
cleanup is blockeddependency inventory and owning serviceremove dependants in reviewed reverse order

Keep the original symptom and timestamp. State one hypothesis, make one reversible change, repeat the original test, and record rollback. Never open a management port to the world, expose credentials, disable TLS verification, format an unknown disk, or add broad permissions as a generic fix.

Architecture and certification decisions

Professional-level questions provide competing valid features. Identify the requirement that decides between them: human or workload identity, same-account or cross-account access, regional or zonal scope, stateful or stateless filtering, durable or ephemeral data, latency, RTO/RPO, cost, or operational ownership.

Explain why the selected option fits and why each plausible alternative fails one stated requirement. Do not rely on feature memorization or reproduce protected certification questions.

Knowledge check

  1. Which check normally indicates host infrastructure trouble?

Expected direction: System status check.

  1. Which check detects guest network or kernel problems?

Expected direction: Instance status check.

  1. Does automatic recovery handle an unhealthy web process?

Expected direction: No. It responds to eligible system-status failure, not ordinary application health.

  1. What does recovery lose even when identity is preserved?

Expected direction: Volatile RAM and operating-system uptime; applicable instance-store considerations also matter.

Cost, cleanup, and retained state

Retain only the state explicitly required by the next P04 lesson and record it in the private resource ledger.

Cleanup evidence includes the final state query, not only a successful delete response. Search related resources, other Regions used by the lab, retained storage, public IPv4 addresses, logging destinations, and service-managed dependencies. Schedule a later billing review because cost data can lag.

Completion gate

Pass when the practical artifact explains the real problem, matches Console and CLI evidence, answers every knowledge check, diagnoses one failure without broadening access, records cost, and proves cleanup or approved retention. The learner must defend one decision orally and name the requirement that would change it.

Official sources

Advertisement