AWS 145: Amazon SQS queues
Why this lesson matters
Decouple producers and consumers using Standard or FIFO queues with correct visibility, long polling, retention, encryption, DLQ, and idempotent processing.
What you will be able to do
By the end, you can:
- explain amazon sqs queues in plain language;
- locate the current service controls in the AWS Management Console;
- run the matching CloudShell or AWS CLI queries and explain every important field;
- draw the identity, network, data, failure, and monitoring path;
- choose the service from requirements and reject it when those requirements are absent;
- diagnose a failed or misleading result from evidence;
- state the cost owner and prove cleanup or a no-create result.
Before you start
- Use a personal AWS account only when its owner has approved the lesson. Do not use the root user for daily work.
- CloudShell is the default command environment. AWS028 explains CloudShell; AWS029 and AWS030 explain local AWS CLI installation and profiles.
- The course example Region is
ap-south-1. Global services and services with a required control Region are called out in their commands. - Run
aws sts get-caller-identityprivately. Redact the account number before sharing evidence. - Never paste access keys, passwords, secret values, private object data, presigned URLs, or full account-specific ARNs into a submission.
- This is a no-create lesson. Every Console action and AWS CLI command is read-only. Create the practical artifact locally.
- Console wording can change. Use the Console service search if a menu label has moved, then confirm the current field in the official documentation.
The core model
| Question | What it means in this lesson |
|---|---|
| Purpose | Decouple producers and consumers using Standard or FIFO queues with correct visibility, long polling, retention, encryption, DLQ, and idempotent processing. |
| Scope and boundary | The learner must identify the account and Region scope, resource boundary, identity path, data or network path, failure behavior, observability, and cleanup ownership for Amazon SQS. |
| Evidence of success | Success means the Console fields, CLI result, workload behavior, monitoring evidence, and architecture claim agree. An available state alone is not enough for Amazon SQS. |
| Cost model | Requests, running capacity, storage, logs, data transfer, retained state, and optional features must be priced for the exact design. |
| Safe rejection rule | Avoid assuming Standard queues deliver exactly once or setting visibility shorter than realistic processing without extension. |
How the request flows
+----------------------+
| Producer sends |
+----------------------+
|
v
+----------------------+
| Durable queue |
+----------------------+
|
v
+-------------------------------+
| Consumer receive and delete |
+-------------------------------+
|
v
+---------------------------------------+
| Retry, visibility, and DLQ evidence |
+---------------------------------------+For Amazon SQS, the important boundary is this: The learner must identify the account and Region scope, resource boundary, identity path, data or network path, failure behavior, observability, and cleanup ownership for Amazon SQS. Success means the Console fields, CLI result, workload behavior, monitoring evidence, and architecture claim agree. An available state alone is not enough for Amazon SQS. That is why the lesson pairs the Console with CLI output and a practical artifact. One interface may hide a field, use a cached view, or be scoped differently. Matching evidence is stronger than a screenshot alone.
Architecture decision table
| Situation | Direction | Reason |
|---|---|---|
| Requirement matches | Use SQS to buffer work, isolate failure, and let consumers process at their own safe rate. | Select only after scope, behavior, security, recovery, operations, and price evidence agree. |
| Requirement does not match | Avoid assuming Standard queues deliver exactly once or setting visibility shorter than realistic processing without extension. | Rejecting an attractive service is a valid architecture result. |
| No create permission or cost approval | Use supplied evidence and local design work | Learning does not depend on creating an hourly resource. |
| Existing resource is unknown or unowned | Inspect only, then stop | Never change or delete a resource merely because it resembles a course example. |
Queue lifecycle and guarantees
A producer sends a message and receives an ID when SQS accepts it. A consumer receives the message plus a temporary receipt handle. It remains stored but invisible until visibility expires. Successful processing must be followed by deletion using the current receipt handle; receive is not acknowledgement. A crash, timeout or lost delete makes it visible again. Standard queues are high-throughput, at-least-once and best-effort ordered, so durable business idempotency is mandatory.
FIFO adds ordered processing within each MessageGroupId and send deduplication using an explicit or content-based ID in the deduplication interval. Group cardinality controls parallelism: one group serializes its messages. FIFO transport still does not make a payment/database side effect globally exactly once; conditionally claim the business key.
Set visibility above high-percentile processing plus delete/network margin, or heartbeat with bounded ChangeMessageVisibility. Too short causes duplicates; too long delays recovery. Use long polling to reduce empty requests. Delay controls first visibility; retention controls survival. Put oversized content in S3 and queue a protected pointer/checksum after validating current message-size limits.
A redrive policy moves repeatedly received messages to a compatible DLQ. The DLQ needs retention, alarm, owner and controlled redrive. DLQ movement is business-flow failure, not success. Queue/IAM policy, KMS policy/grants and VPC endpoint policy are distinct. Monitor approximate visible/not-visible/delayed counts, oldest age, sent/received/deleted/empty receives, DLQ depth and application outcomes.
AWS Management Console, step by step
Sign in with the normal non-root learning identity. Write the expected starting state before opening the service.
- Use the Console service search and open SQS, Queues; confirm the account and Region before reading the page.
- Inspect the supplied or owned resource's status, configuration, permissions, networking, encryption, monitoring, tags, and dependencies without changing it.
- Open the related metrics, logs, events, or history view and record one timestamped signal that would prove or disprove the expected behavior.
- Return to the resource list, clear filters, and record the final inventory. On the read-only track, do not choose Create, Save, or Delete.
CloudShell and AWS CLI, step by step
Start with a known caller and Region:
export AWS_DEFAULT_REGION="ap-south-1"
aws sts get-caller-identity --query Arn --output text
aws configure listRedact the account part of the ARN in shared evidence. Now run the topic queries:
aws sqs list-queues --output table
QUEUE_URL='replace-with-owned-queue-url'
aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --attribute-names All --output jsonExpected interpretation
Queue attributes describe policy and timers. Approximate counts are not a transaction ledger, and successful receive does not mean successful business processing.
Practical work
Design an order queue with message schema, Standard or FIFO choice, group and deduplication keys if needed, visibility timeout, long polling, retention, redrive count, DLQ alarm, and idempotency key.
Calculate drain time for 100,000 messages at stated batch/concurrency/duration and stop at downstream capacity. Test duplicate delivery, crash after commit before delete, visibility expiry, poison message, DLQ transition/redrive, retention expiry, KMS denial and unauthorized producer. Include schema version, correlation ID, sensitive-data rule and deletion/purge safeguards.
Diagnose this topic from its own evidence
Growing oldest age and visible depth means arrivals exceed successful deletions. High not-visible with low deletes suggests slow/stuck consumers or excessive visibility; duplicate side effects plus receive count indicates expiry/crash and missing idempotency; empty receives indicate polling/scope mismatch. Inspect message ID, business ID, receive count, timestamp and consumer logs. Never purge an unknown queue during diagnosis.
Cost and cleanup
Requests, running capacity, storage, logs, data transfer, retained state, and optional features must be priced for the exact design.
Knowledge check
- What operational purpose is this lesson solving?
Expected direction: Decouple producers and consumers using Standard or FIFO queues with correct visibility, long polling, retention, encryption, DLQ, and idempotent processing.
- Which scope or ownership boundary must be proved first?
Expected direction: The learner must identify the account and Region scope, resource boundary, identity path, data or network path, failure behavior, observability, and cleanup ownership for Amazon SQS.
- What evidence is strong enough to accept the result?
Expected direction: Success means the Console fields, CLI result, workload behavior, monitoring evidence, and architecture claim agree. An available state alone is not enough for Amazon SQS.
- Which tempting design or shortcut must be rejected?
Expected direction: Avoid assuming Standard queues deliver exactly once or setting visibility shorter than realistic processing without extension.
- Which cost dimensions and retained resources need an owner?
Expected direction: Requests, running capacity, storage, logs, data transfer, retained state, and optional features must be priced for the exact design.
Lesson acceptance
Pass when the learner explains send/receive/visibility/delete, Standard/FIFO order/dedup/group throughput, long polling, retention/delay, security, DLQ/redrive and approximate metrics. Numeric capacity/visibility, idempotency, poison/replay tests, alarms, costs and cleanup are mandatory. Fail if receive equals acknowledgement, FIFO replaces business idempotency, or a DLQ lacks monitoring and recovery ownership.