Using Presing in AWS
Presing is a command you can use in the AWS CLI that allows anyone to have the pre-signed URL to make and HTTP get request to retrieve the data that is inside the bucket pre-signed. In the CLI you will execute:
aws s3 presign s3://DOC-EXAMPLE-BUCKET/test2.tx
t
And the output you’re going to receive is the following:
https://DOC-EXAMPLE-BUCKET.s3.us-west-2.amazonaws.com/key?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAEXAMPLE123456789%2F20210621%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210621T041609Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=EXAMBLE1234494d5fba3fed607f98018e1dfc62e2529ae96d844123456
Then you can do an HTTP get request or, for example, load CSV for NEO4J:
LOAD CSV WITH HEADERS FROM "PRESING URL" as row return count(row)
One interesting feauture about presing is the possibilit to reduce the time the URL generated is exposed. For that, you have to use the command --expires-in
The important values are:
- Default is 3600 seconds = 60 minutes
- The maximum is 604800 seconds = 10080 minutes = 168 hours = 7 days
- The minimum is 300 seconds = 5 minutes
Example:
aws s3 presign s3://DOC-EXAMPLE-BUCKET/test2.txt --expires-in
300 You can also presing from python. For example:import boto3 url = boto3.client('s3').generate_presigned_url( ClientMethod='get_object', Params={'Bucket': 'BUCKET_NAME', 'Key': 'OBJECT_KEY'}, ExpiresIn=3600)
Official documentation: