Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days AgroAnalytics Airflow Amazon Web Services AWS Azure Batch code Coders At Work Computer Science CSS Data Databases DataScience Django Edsger W. Dijkstra Excel Finantial Data Google Hellerstein Javascript Jinja Knowledge Transfer MVC Networking Networks Protocols Notes on Structured programming People Analytics PHP Power Bi Powershell Python Redash Social Skills Software Developer Spark: The Definitive Guide SQL SSH Testing The Elements of Programming Style The humble programmer Traversy Media Understanding Distributed Systems Web Development Yaml

Archives

Category: Cloud

Understanding Stateful vs. Stateless Firewalls in AWS

When working with network security, it’s crucial to understand the difference between stateful and stateless firewalls. In AWS, this understanding is particularly important when configuring security groups and network access control lists (ACLs). Stateless Firewalls A stateless firewall handles each packet independently, without remembering any information about previous packets. This means that you must explicitly
Read More

Handling shutil.SameFileError When Copying Files in Python

When using Python’s shutil.copy() or shutil.copy2() to copy files, you might run into a shutil.SameFileError if you mistakenly attempt to copy a file onto itself. This error occurs when the source and destination paths are the same, disrupting your script’s execution. In this post, we’ll explore how to prevent this error and ensure files are
Read More

Preserving Directory Structure While Copying Files in Python – version 2

When copying files from one directory to another in Python, it’s important to maintain the original directory structure, especially when dealing with nested directories. In this post, we’ll explore how to use Python’s shutil and os libraries to copy files while preserving the directory structure. Problem Imagine you have a source directory with nested folders
Read More

Avoiding Duplicate File Copies Based on Content in Python on AWS

When working with large file systems, copying files can often lead to unintentional duplication, especially if files with the same content are repeatedly copied into different directories. While filenames can vary, the underlying content might remain the same, leading to redundant data and wasted storage space. In this post, we’ll explore how to avoid copying
Read More

Handling NoneType Errors When Extending Lists in Python

When working with Python, especially with functions that return lists or other iterable objects, you might encounter a TypeError that says something like: This error occurs when you try to iterate over or extend a list using a value that turns out to be None. In Python, NoneType represents a null value, and it is
Read More

Tracking File Changes in S3 Using ETags

When working with AWS S3, tracking changes to files can be essential, especially when versioning is not enabled on the bucket. The ETag associated with each file in S3 can provide a simple way to detect changes. In this post, we’ll explore how to use ETags to monitor file modifications in an S3 bucket. What
Read More

Efficiently Listing and Filtering S3 Objects by Date

When working with AWS S3 buckets, it’s common to have a large number of objects stored, and you might need to filter them based on certain criteria like dates. This blog post will guide you on how to efficiently list and filter S3 objects by date using Python and the boto3 library. Why Filtering by
Read More