Concepts

Event source mapping is what directs events from your source (DynamoDB, Kinesis, or SQS) to an AWS Lambda function. When an event occurs in your event source, it triggers the Lambda function to execute. This is particularly useful for creating reactive, event-driven architectures within AWS.

Types of Event Sources

AWS Lambda supports several types of event sources:

  • Stream-based sources: Such as Amazon Kinesis and DynamoDB Streams.
  • Queue-based sources: Such as Amazon SQS.
  • Other: AWS Lambda also natively supports S3 events, SNS notifications, and many more, but the event source mapping is primarily for streams and queues.

Basic Configuration for Event Source Mapping

To set up an event source mapping via the AWS CLI, use the create-event-source-mapping command. The following parameters are essential:

  • --event-source-arn: The ARN of the event source (stream or queue).
  • --function-name: The name of the Lambda function you want to invoke.
  • --starting-position: Only for stream sources; it describes where to start reading events (e.g., LATEST or TRIM_HORIZON).

For streams (DynamoDB or Kinesis):

aws lambda create-event-source-mapping \
--event-source-arn arn:aws:kinesis:REGION:ACCOUNT_ID:stream/STREAM_NAME \
--function-name FUNCTION_NAME \
--starting-position LATEST

For SQS queues:

aws lambda create-event-source-mapping \
--event-source-arn arn:aws:sqs:REGION:ACCOUNT_ID:queue/QUEUE_NAME \
--function-name FUNCTION_NAME

Best Practices with Event Source Mapping

When using event source mapping, there are some best practices to keep in mind:

  • Batch Size Configuration: Configure the batch size according to your workload. For Kinesis and DynamoDB Streams, this can be up to 10,000 records or 10 MB, whichever is smaller. For SQS, it can be up to 10 messages.
  • Error Handling: Make sure to handle errors in your Lambda function so that unsuccessful processing doesn’t block the stream or queue.
  • Parallelization: With Kinesis, you can set up parallelization by setting the --parallelization-factor.
  • DLQ Configuration: For SQS event sources, set up a Dead Letter Queue (DLQ) to capture any messages that couldn’t be processed successfully after multiple attempts.

Limitations and Considerations

Keep in mind there are limits and trade-offs:

  • Execution Role Permissions: Your Lambda function’s execution role must have the necessary permissions to poll the event source and invoke your function.
  • Concurrency: Stream-based triggers preserve the order of records, queue-based do not necessarily do so.
  • Scaling: Stream-based event sources scale the number of Lambda function invocations based on the number of shards in the stream. For SQS, it’s based on the number of messages in the queue.

Monitoring and Troubleshooting

AWS provides monitoring capabilities for your event source mappings through Amazon CloudWatch. You can monitor metrics like:

  • Invocation counts
  • Error counts
  • Throttles
  • IteratorAge for streams, which measures the age of the last record for each batch.

When configured correctly, event source mapping is a powerful tool in AWS that allows seamless integration between different AWS services and Lambda, but it’s critical to understand the characteristics of each event source type, their limitations, and how to troubleshoot common issues.

As you prepare for your AWS Certified Developer – Associate exam, ensure you’re comfortable creating, updating, and troubleshooting event source mappings as these are practical, hands-on skills that are likely to be tested.

Answer the Questions in Comment Section

(True/False) Event source mapping is only used with AWS Lambda to poll Kinesis data streams.

  • True
  • False

Answer: False

Explanation: Event source mapping is used to connect AWS Lambda functions to various data sources such as Amazon Kinesis Data Streams, Amazon DynamoDB Streams, and Amazon SQS queues, not just Kinesis data streams.

(Single Select) Which AWS service is NOT directly integrated with AWS Lambda using event source mapping?

  • Amazon Kinesis
  • Amazon S3
  • Amazon MQ
  • Amazon DynamoDB

Answer: Amazon S3

Explanation: Amazon S3 is integrated with AWS Lambda through a different mechanism involving S3 event notifications, not through event source mapping.

(True/False) You can create an event source mapping using the AWS Management Console, AWS CLI, or the AWS SDKs.

  • True
  • False

Answer: True

Explanation: AWS provides flexibility to create an event source mapping through various interfaces such as the AWS Management Console, AWS CLI, and AWS SDKs.

(Multiple Select) Which types of processing can event source mappings help facilitate with AWS Lambda?

  • Batch processing
  • Real-time processing
  • Transactional processing
  • Stream processing

Answer: Batch processing, Real-time processing, Stream processing

Explanation: Event source mappings can facilitate batch processing, real-time processing, and stream processing. It helps in batching records when polling data sources before invoking the Lambda function.

(True/False) Batch size for an event source mapping with Kinesis Data Streams is configurable up to 10,000 records or 6 MB.

  • True
  • False

Answer: True

Explanation: For Kinesis Data Streams, you can configure the batch size up to 10,000 records, or a total payload size of 6 MB, whichever is reached first.

(Single Select) Event source mapping with DynamoDB streams can trigger a Lambda function:

  • Only when new items are added to the table.
  • When items are updated, added, or deleted in the table.
  • When the table is deleted.
  • None of the above.

Answer: When items are updated, added, or deleted in the table.

Explanation: DynamoDB Streams can trigger a Lambda function for various table activities, including item updates, additions, and deletions.

(True/False) Once event source mapping is created, it cannot be updated.

  • True
  • False

Answer: False

Explanation: Event source mappings can be updated to change configurations such as batch size, starting position, and enabled/disabled status.

(Multiple Select) What are some best practices when using event source mapping with AWS Lambda?

  • Setting appropriate batch sizes
  • Processing records in the order they are received
  • Deleting records from the stream
  • Retrying failed batches

Answer: Setting appropriate batch sizes, Processing records in the order they are received, Retrying failed batches

Explanation: Setting appropriate batch sizes and retrying failed batches are best practices to ensure efficient Lambda processing. It is also important to process records in the order they are received, especially with Kinesis and DynamoDB Streams. You should not delete records from the stream, as this is managed by the service itself.

(True/False) AWS ensures that events are delivered exactly once and in the order they are generated when using SQS with event source mapping.

  • True
  • False

Answer: False

Explanation: AWS SQS queues do not guarantee exactly-once delivery or order of messages. However, FIFO (First-In-First-Out) queues aim to ensure order and exactly-once processing to some extent.

(Single Select) What is the initial position setting used in event source mapping with Kinesis Data Streams and DynamoDB Streams?

  • LATEST
  • TRIM_HORIZON
  • AT_TIMESTAMP
  • SHARD_END

Answer: TRIM_HORIZON

Explanation: TRIM_HORIZON is a common initial position setting used in event source mappings that tells the service to start processing from the oldest record available in the stream.

(True/False) You must manually configure an event source mapping’s error handling, including setting up a dead-letter queue.

  • True
  • False

Answer: True

Explanation: Error handling must be configured manually for event source mappings. You can set up a dead-letter queue (DLQ) to manage Lambda invocation errors.

(True/False) The number of concurrent batches per shard is limited when using event source mapping with AWS Lambda and Amazon Kinesis.

  • True
  • False

Answer: True

Explanation: When using event source mapping with Amazon Kinesis, there is a limit of 5 concurrent batches per shard. This is to prevent read throttling and potential data loss.

0 0 votes
Article Rating
Subscribe
Notify of
guest
24 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Catalina Muñoz
7 months ago

Can anybody explain Event Source Mapping with DynamoDB Streams in a bit more detail?

Emmanuel Holberg
7 months ago

Appreciate the information presented in this blog post!

Lillie Green
7 months ago

Does AWS provide any monitoring tools for Event Source Mapping?

Champak Naik
7 months ago

The article was good. Keep up the great work!

Morgane Fontai
7 months ago

Just what I was looking for! Thanks!

Florian Picard
6 months ago

Great blog post! I was struggling with Event Source Mapping, and this cleared up a lot of my confusion.

Miguel Benavides
7 months ago

Can someone explain the permissions required for setting up Event Source Mapping with Lambda?

Josette Perez
7 months ago

Thanks for the detailed insights!

24
0
Would love your thoughts, please comment.x
()
x