Sending Athena Query Results to Amazon SQS: Architecture, Costs, and Limitations
Introduction
Amazon Athena is a serverless query service designed for interactive analysis of data stored in Amazon S3. Amazon SQS (Simple Queue Service), on the other hand, is a fully managed message queue used to decouple and scale distributed systems. A common question arises when combining both services: can Athena send query results directly to SQS? The short answer is no. This article explains why, what the correct architecture looks like, and the associated costs.
Why Athena Cannot Send Data Directly to SQS
Amazon Athena is a read-only query engine. Its execution model is limited to:
- Reading data from S3
- Writing query results back to S3
Athena has no native integration to push data into downstream services such as SQS, SNS, DynamoDB, or Kinesis. This is a deliberate design decision aligned with its role as an analytical service rather than an event producer.
Reference:
https://docs.aws.amazon.com/athena/latest/ug/what-is.html
The Correct Architecture Pattern
To move data from Athena to SQS, an intermediate step is required.
Standard Flow
- Athena executes a SQL query.
- Query results are written to Amazon S3.
- A compute component reads the results from S3.
- Each record is published as a message to Amazon SQS.
Common Implementations
- AWS Lambda triggered by S3
ObjectCreatedevents - AWS Glue jobs (Python Shell or Spark)
- External workers using boto3
- AWS Step Functions orchestrating the workflow
This pattern respects AWS service boundaries:
- Athena for analytics
- S3 for durable storage
- SQS for asynchronous message delivery
Reference:
https://docs.aws.amazon.com/athena/latest/ug/querying.html
Cost Considerations
Amazon Athena
- Billed per data scanned: USD 5 per TB
- Output data stored in S3 is billed separately
Reference:
https://aws.amazon.com/athena/pricing/
Amazon SQS
- 1,000,000 requests per month are free
- After the free tier:
- Standard queues: approximately USD 0.40 per million requests
- Each send, receive, and delete operation counts as a request
- Messages larger than 64 KB are billed as multiple requests
For moderate batch workloads (for example, tens of thousands of messages per month), SQS usage typically remains within the free tier.
Reference:
https://aws.amazon.com/sqs/pricing/
Message Retention and Delivery Semantics
Messages sent to Amazon SQS:
- Remain in the queue until explicitly deleted by a consumer
- Or until the Message Retention Period expires (configurable from 1 minute to 14 days, default 4 days)
- Are delivered with at-least-once semantics in Standard queues
- May be delivered more than once unless FIFO queues are used
When This Architecture Is Appropriate
This pattern is suitable when:
- Analytical results need to trigger asynchronous processing
- Batch analytics must be decoupled from downstream systems
- Reliability and retry semantics are required
It is not suitable for:
- Real-time streaming use cases
- High-frequency event generation directly from SQL queries
Conclusion
Amazon Athena cannot send data directly to Amazon SQS. The supported and scalable approach is to use Amazon S3 as an intermediate storage layer and a compute service (such as AWS Lambda or AWS Glue) to publish messages to SQS. This design aligns with AWS best practices, keeps costs predictable, and scales reliably.