Sqs

Amazon SQS managed message queue service.

by claude-dev-suite · MIT

The skill

Amazon SQS managed message queue service. Covers standard and FIFO queues, dead-letter queues, and integration patterns. Use for AWS-native serverless and microservices architectures. USE WHEN: user mentions "sqs", "aws queues", "fifo queue", "lambda trigger", "sns to sqs", asks about "aws messaging", "serverless queues", "standard queue", "visibility timeout" DO NOT USE FOR: event streaming - use `kafka` or AWS Kinesis; Azure-native - use `azure-service-bus`; GCP-native - use `google-pubsub`; on-premise - use `rabbitmq` or `activemq`; complex routing - use `rabbitmq`

Amazon SQS Core Knowledge

Full Reference: See advanced.md for Java/Python/Go producers, Spring Cloud AWS consumers, Lambda integration, IAM policies, and CloudWatch monitoring.

Deep Knowledge: Use mcp__documentation__fetch_docs with technology: sqs for comprehensive documentation.

Quick Start (LocalStack)

# docker-compose.yml
services:
  localstack:
    image: localstack/localstack
    ports:
      - "4566:4566"
    environment:
      - SERVICES=sqs
      - DEFAULT_REGION=us-east-1
    volumes:
      - localstack_data:/var/lib/localstack

volumes:
  localstack_data:

# Create queue
aws --endpoint-url=http://localhost:4566 sqs create-queue \
  --queue-name orders-queue

# Create FIFO queue
aws --endpoint-url=http://localhost:4566 sqs create-queue \
  --queue-name orders-queue.fifo \
  --attributes FifoQueue=true,ContentBasedDeduplication=true

Core Concepts

Concept	Description
Standard Queue	At-least-once, best-effort ordering
FIFO Queue	Exactly-once, strict ordering
Visibility Timeout	Message lock period
Dead Letter Queue	Failed message destination
Long Polling	Efficient message retrieval
Message Groups	FIFO ordering within group

Queue Types Comparison

Feature	Standard	FIFO
Throughput	Unlimited	300 msg/s (3000 with batching)
Ordering	Best-effort	Strict (per group)
Delivery	At-least-once	Exactly-once
Deduplication	None	5-minute window

Producer Pattern (Node.js)

import { SQSClient, SendMessageCommand, SendMessageBatchCommand } from '@aws-sdk/client-sqs';

const client = new SQSClient({
  region: 'us-east-1',
  // For LocalStack: endpoint: 'http://localhost:4566',
});

const queueUrl = 'https://sqs.us-east-1.amazonaws.com/123456789/orders-queue';

// Send single message
await client.send(new SendMessageCommand({
  QueueUrl: queueUrl,
  MessageBody: JSON.stringify(order),
  MessageAttributes: {
    'OrderType': {
      DataType: 'String',
      StringValue: order.type,
    },
    'CorrelationId': {
      DataType: 'String',
      StringValue: correlationId,
    },
  },
  DelaySeconds: 0,
}));

// Send batch (up to 10)
await client.send(new SendMessageBatchCommand({
  QueueUrl: queueUrl,
  Entries: orders.map((order, index) => ({
    Id: `msg-${index}`,
    MessageBody: JSON.stringify(order),
    MessageAttributes: {
      'OrderType': { DataType: 'String', StringValue: order.type },
    },
  })),
}));

// FIFO queue
const fifoQueueUrl = 'https://sqs.us-east-1.amazonaws.com/123456789/orders-queue.fifo';

await client.send(new SendMessageCommand({
  QueueUrl: fifoQueueUrl,
  MessageBody: JSON.stringify(order),
  MessageGroupId: order.customerId,      // Required for FIFO
  MessageDeduplicationId: order.orderId, // Or use ContentBasedDeduplication
}));

Consumer Pattern (Node.js)

import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';

const client = new SQSClient({ region: 'us-east-1' });

async function pollMessages() {
  while (true) {
    const response = await client.send(new ReceiveMessageCommand({
      QueueUrl: queueUrl,
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20,           // Long polling
      VisibilityTimeout: 30,
      MessageAttributeNames: ['All'],
      AttributeNames: ['All'],
    }));

    if (!response.Messages) continue;

    for (const message of response.Messages) {
      try {
        const order = JSON.parse(message.Body!);
        await processOrder(order);

        // Delete on success
        await client.send(new DeleteMessageCommand({
          QueueUrl: queueUrl,
          ReceiptHandle: message.ReceiptHandle!,
        }));
      } catch (error) {
        // Message will return to queue after visibility timeout
        console.error('Processing failed:', error);
      }
    }
  }
}

Dead Letter Queue

// Create DLQ
await client.send(new CreateQueueCommand({
  QueueName: 'orders-dlq',
}));

// Main queue with DLQ
await client.send(new CreateQueueCommand({
  QueueName: 'orders-queue',
  Attributes: {
    RedrivePolicy: JSON.stringify({
      deadLetterTargetArn: 'arn:aws:sqs:us-east-1:123456789:orders-dlq',
      maxReceiveCount: '3',
    }),
    VisibilityTimeout: '30',
    MessageRetentionPeriod: '1209600', // 14 days
  },
}));

# Terraform
resource "aws_sqs_queue" "orders_dlq" {
  name = "orders-dlq"
  message_retention_seconds = 1209600
}

resource "aws_sqs_queue" "orders" {
  name = "orders-queue"
  visibility_timeout_seconds = 30
  message_retention_seconds = 1209600

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.orders_dlq.arn
    maxReceiveCount     = 3
  })
}

Lambda Integration

// Lambda handler
export const handler = async (event: SQSEvent): Promise<SQSBatchResponse> => {
  const batchItemFailures: SQSBatchItemFailure[] = [];

  for (const record of event.Records) {
    try {
      const order = JSON.parse(record.body);
      await processOrder(order);
    } catch (error) {
      // Report partial batch failure
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }

  return { batchItemFailures };
};

# serverless.yml
functions:
  orderProcessor:
    handler: handler.handler
    events:
      - sqs:
          arn: !GetAtt OrdersQueue.Arn
          batchSize: 10
          functionResponseType: ReportBatchItemFailures

When NOT to Use This Skill

Use alternative messaging solutions when:

Event streaming with replay - Use Kinesis or Kafka
Cross-cloud or hybrid cloud - Use Kafka, RabbitMQ, or Pulsar
Complex routing patterns - RabbitMQ provides better routing
Message ordering across all messages - FIFO queues have throughput limits
Real-time low latency (<10ms) - Use Redis or NATS
On-premise deployment - Use RabbitMQ or ActiveMQ
Message size >256KB - Use S3 with SQS Extended Client

Anti-Patterns

Anti-Pattern	Why It's Bad	Solution
Short polling (WaitTime=0)	Wasteful API calls, higher cost	Use long polling (WaitTimeSeconds=20)
No DLQ configured	Failed messages lost	Always configure dead letter queue
Visibility timeout too short	Duplicate processing	Set timeout > max processing time
Visibility timeout too long	Slow failure recovery	Balance with processing time
Processing before delete	Message reprocessed on crash	Delete only after successful processing
FIFO for high throughput	Limited to 300 msg/s	Use Standard queue if ordering not critical
No batching	Higher latency and cost	Batch up to 10 messages per request
Polling in Lambda	Wasted invocations	Use Lambda event source mapping
No IAM policies	Security risk	Apply least privilege IAM policies

Quick Troubleshooting

Issue	Likely Cause	Fix
Messages not appearing	Wrong queue URL or permissions	Verify URL and IAM permissions
Duplicate messages	Standard queue behavior or visibility timeout	Implement idempotent processing
Messages delayed	Delay seconds set or queue backlog	Check DelaySeconds, scale consumers
Messages in DLQ	Max receives exceeded	Check processing logic, increase retries
Visibility timeout errors	Message processing too slow	Extend visibility timeout
Throughput limited	FIFO queue limit	Use Standard queue or batch messages
High costs	Short polling or frequent sends	Use long polling, batch operations
Access denied	Missing IAM permissions	Add sqs:SendMessage/ReceiveMessage
Message size limit	Payload >256KB	Use SQS Extended Client with S3

Production Checklist

[ ] IAM policies with least privilege
[ ] Server-side encryption enabled
[ ] Dead letter queue configured
[ ] Visibility timeout set appropriately
[ ] Long polling enabled (20s)
[ ] Message retention configured
[ ] CloudWatch alarms set up
[ ] DLQ monitoring alerts
[ ] VPC endpoints (if needed)
[ ] Access logging enabled

Key Metrics to Monitor

Metric	Alert Threshold
ApproximateNumberOfMessagesVisible	> 10000
ApproximateAgeOfOldestMessage	> 3600s
NumberOfMessagesReceived	Anomaly
NumberOfMessagesSent	Anomaly
ApproximateNumberOfMessagesNotVisible	> expected

Reference Documentation