Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql PostHog Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Understanding Distributed Systems whale-alert Yaml

Archives

OLTP vs. OLAP: How JOINs and Efficiency Shape Their Differences

Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) are two distinct database architectures, each designed for different purposes. One key factor that differentiates them is how they handle JOIN operations and the impact these have on query performance. In this post, we’ll explore these differences and why OLAP tends to be more efficient for
Read More

The Origins of OLTP and OLAP: A Brief History

Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) are fundamental concepts in database management, each serving distinct purposes. But when did these terms first appear, and how did they evolve? Let’s explore their origins and how they became the cornerstone of modern data systems. The Emergence of OLTP The concept of Online Transaction Processing
Read More

Comparison Between Star Schema and Snowflake Schema in PostgreSQL

Comparison Between Star Schema and Snowflake Schema in PostgreSQL When designing a database for analytical workloads, choosing the right schema can significantly impact performance and query efficiency. The two most common data warehouse schema models are Star Schema and Snowflake Schema. In this post, we’ll explore the differences between these schemas, their advantages and disadvantages,
Read More

Testing Apache Airflow DAGs: A Modular Approach

Introduction Apache Airflow is a powerful workflow automation tool, but testing DAGs can be challenging due to their dependency on the Airflow scheduler and execution environment. In this post, we explore a modular approach that enables easier testing by extracting logic into separate Python functions, which can be tested independently. Methods for Testing Airflow DAGs
Read More

Running PySpark on Google Colab: Do You Still Need findspark?

Introduction For a long time, using Apache Spark in Google Colab required manual setup, including installing Spark and configuring Python to recognize it. This was often done using the findspark library. However, recent changes in Colab have made this process much simpler. In this post, we will explore whether findspark is still necessary and the
Read More

Visualizing EXPLAIN ANALYZE in PostgreSQL

When working with PostgreSQL, understanding how queries execute can greatly improve performance tuning and optimization. PostgreSQL provides the EXPLAIN ANALYZE command to help developers analyze query execution plans. However, reading raw execution plans can be challenging. In this post, we will explore how to interpret EXPLAIN ANALYZE output and visualize it using online tools. Understanding
Read More

Understanding Subnets, Gateways, and Route Tables in AWS

When designing applications in AWS, it’s crucial to understand how networking components interact within a Virtual Private Cloud (VPC). This post will cover subnets, gateways, and route tables, specifically in the context of the AWS Free Tier. Everything Lives Inside a VPC A VPC (Virtual Private Cloud) is a logically isolated section of AWS where
Read More

Enabling Internet Access for Resources in a Public Subnet

When deploying resources in a public subnet within an AWS Virtual Private Cloud (VPC), you need to configure several components to allow them to communicate with the internet. Below are the essential steps: 1. Attach an Internet Gateway (IGW) An Internet Gateway (IGW) enables communication between instances in your VPC and the internet. To set
Read More

Network Address Translation (NAT): Overcoming IPv4 Shortages

Introduction Network Address Translation (NAT) is a technology designed to mitigate the shortage of IPv4 addresses by allowing multiple devices on a private network to share a limited number of public IP addresses. This process involves translating private IPv4 addresses to public addresses, enabling seamless communication with external networks. Types of NAT There are three
Read More

Splitting Strings in Excel: A Simple Guide

When working with Excel, you may encounter situations where you need to split a string into separate parts. For example, consider the following string: If you want to separate orderId and the actual ID value, there are several methods available depending on your Excel version. Method 1: Using TEXTSPLIT (Excel 365/2021) If you’re using the
Read More