Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Traversy Media Understanding Distributed Systems Virtual Box whale-alert Yaml

Archives

Tracking File Changes in S3 Using ETags

When working with AWS S3, tracking changes to files can be essential, especially when versioning is not enabled on the bucket. The ETag associated with each file in S3 can provide a simple way to detect changes. In this post, we’ll explore how to use ETags to monitor file modifications in an S3 bucket. What
Read More

Implementing Retries in Python

In many real-world applications, simply handling an error isn’t always enough. Sometimes, the failure is temporary, and retrying the operation can help resolve the issue. In this post, we’ll explore how to implement retries in Python, improving the robustness of our programs. Why Implement Retries? Let’s imagine you’re making a request to an external API
Read More

Efficiently Listing and Filtering S3 Objects by Date

When working with AWS S3 buckets, it’s common to have a large number of objects stored, and you might need to filter them based on certain criteria like dates. This blog post will guide you on how to efficiently list and filter S3 objects by date using Python and the boto3 library. Why Filtering by
Read More

Handling Split Errors in Azure Data Factory: A Step-by-Step Guide

In Azure Data Factory (ADF), we often use expressions to manipulate strings and extract specific parts of data. One common operation is splitting strings based on a delimiter. However, this can sometimes lead to errors, especially if the string format is not consistent. In this blog post, we’ll explore a common issue with the split
Read More

Customizing Legends in Seaborn Boxplots: A Guide

Creating clear and informative visualizations is key to effectively communicating data insights. In this post, we will explore how to customize legends in Seaborn boxplots, ensuring that the labels and colors are both informative and accurate. Specifically, we’ll look at how to manually set the legend labels while maintaining the correct color associations in a
Read More

Comparing Window Functions with Aggregate Functions in SQL

Introduction SQL is a powerful language for querying and manipulating data, and both window functions and aggregate functions are central to its capabilities. While they serve related purposes, they are used in different contexts and have distinct features. Understanding how window functions and aggregate functions differ is crucial for writing effective SQL queries and performing
Read More

Defining Custom Window Frames in SQL Server

Introduction Window functions in SQL Server are powerful tools that allow for advanced data analysis within queries. One of the key features of window functions is the ability to define custom window frames using the OVER() clause. This customization allows you to specify exactly which rows are included in the calculation, offering flexibility for various
Read More