Daily Failure Reporting in DynamoDB Using Lambda, EventBridge Scheduler, and SES
Operational monitoring requires structured visibility into failures. If your processes write execution logs to DynamoDB and mark failed executions with status = FAILED, you can implement a deterministic daily reporting pipeline using AWS Lambda, EventBridge Scheduler, and Amazon SES.
This article describes a single, production-grade implementation.
Objective
- Source: DynamoDB table containing execution logs
- Filter:
status = FAILED - Frequency: once per day
- Output: email listing failed processes from the previous day
Architecture
- EventBridge Scheduler triggers a Lambda function daily at a fixed local time.
- The Lambda function queries DynamoDB for failed events within the previous calendar day.
- The Lambda formats the results.
- The Lambda sends an email using Amazon SES.
This solution is fully serverless and horizontally scalable.
DynamoDB Data Model
Efficient querying requires correct indexing.
Base Table (minimum attributes)
pk(String): process or execution identifierstatus(String):FAILED,SUCCESS, etc.event_ts(Number): epoch millisecondsmessage(String): optional error message
Global Secondary Index (GSI)
Index name: gsi_status_ts
- Partition key:
status - Sort key:
event_ts
This enables an efficient Query:
status = FAILEDevent_ts BETWEEN start_of_day AND end_of_day
Reference:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html
Lambda Implementation (Python 3.11, boto3)
Environment variables:
DDB_TABLEGSI_NAME(default:gsi_status_ts)SES_FROMSES_TO(comma-separated list)
import os
import time
import datetime
import boto3
from boto3.dynamodb.conditions import KeyDDB_TABLE = os.environ["DDB_TABLE"]
GSI_NAME = os.environ.get("GSI_NAME", "gsi_status_ts")
SES_FROM = os.environ["SES_FROM"]
SES_TO = [s.strip() for s in os.environ["SES_TO"].split(",") if s.strip()]dynamodb = boto3.resource("dynamodb")
ses = boto3.client("ses")def previous_day_range_utc(tz_offset_hours: int = -3) -> tuple[int, int]:
now_utc = datetime.datetime.utcnow()
now_local = now_utc + datetime.timedelta(hours=tz_offset_hours) prev_date = now_local.date() - datetime.timedelta(days=1) start_local = datetime.datetime.combine(prev_date, datetime.time.min)
end_local = datetime.datetime.combine(prev_date, datetime.time.max) start_utc = start_local - datetime.timedelta(hours=tz_offset_hours)
end_utc = end_local - datetime.timedelta(hours=tz_offset_hours) to_ms = lambda dt: int(dt.timestamp() * 1000)
return to_ms(start_utc), to_ms(end_utc)def handler(event, context):
start_ms, end_ms = previous_day_range_utc(tz_offset_hours=-3) table = dynamodb.Table(DDB_TABLE) response = table.query(
IndexName=GSI_NAME,
KeyConditionExpression=Key("status").eq("FAILED") &
Key("event_ts").between(start_ms, end_ms)
) items = response.get("Items", []) while "LastEvaluatedKey" in response:
response = table.query(
IndexName=GSI_NAME,
KeyConditionExpression=Key("status").eq("FAILED") &
Key("event_ts").between(start_ms, end_ms),
ExclusiveStartKey=response["LastEvaluatedKey"]
)
items.extend(response.get("Items", [])) if not items:
body = "No FAILED processes were recorded during the previous day."
else:
lines = []
for item in items:
ts_ms = item.get("event_ts")
ts_str = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(ts_ms / 1000)) if ts_ms else "N/A"
lines.append(f"- {ts_str} | {item.get('pk', 'N/A')} | {item.get('message', '')}") body = "FAILED processes (previous day):\n\n" + "\n".join(lines) ses.send_email(
Source=SES_FROM,
Destination={"ToAddresses": SES_TO},
Message={
"Subject": {"Data": "Daily DynamoDB Failure Report"},
"Body": {"Text": {"Data": body}},
},
) return {"failures_count": len(items)}
DynamoDB Query reference:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html
SES sending reference:
https://docs.aws.amazon.com/ses/latest/dg/send-email-programmatically.html
Scheduling (EventBridge Scheduler)
Create a daily schedule at your required time zone (e.g., America/Argentina/Buenos_Aires) and set the Lambda function as the target.
Reference:
https://docs.aws.amazon.com/eventbridge/latest/userguide/scheduler.html
IAM Permissions
Lambda execution role must allow:
dynamodb:Queryon the GSI resourceses:SendEmail
Scheduler execution role must allow:
lambda:InvokeFunctionon the Lambda function
IAM reference:
https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
Conclusion
A daily failure reporting system in DynamoDB is fundamentally a data modeling problem. Once a proper GSI is defined on status and event_ts, a scheduled Lambda can generate deterministic daily operational reports with minimal infrastructure overhead.
The solution is:
- Fully serverless
- Query-efficient
- Deterministic
- Infrastructure-as-code friendly
This pattern scales without architectural changes and provides structured operational visibility into system failures.