Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Traversy Media Understanding Distributed Systems Virtual Box whale-alert Yaml

Archives

Understanding Subnets, Gateways, and Route Tables in AWS

When designing applications in AWS, it’s crucial to understand how networking components interact within a Virtual Private Cloud (VPC). This post will cover subnets, gateways, and route tables, specifically in the context of the AWS Free Tier. Everything Lives Inside a VPC A VPC (Virtual Private Cloud) is a logically isolated section of AWS where
Read More

Enabling Internet Access for Resources in a Public Subnet

When deploying resources in a public subnet within an AWS Virtual Private Cloud (VPC), you need to configure several components to allow them to communicate with the internet. Below are the essential steps: 1. Attach an Internet Gateway (IGW) An Internet Gateway (IGW) enables communication between instances in your VPC and the internet. To set
Read More

Network Address Translation (NAT): Overcoming IPv4 Shortages

Introduction Network Address Translation (NAT) is a technology designed to mitigate the shortage of IPv4 addresses by allowing multiple devices on a private network to share a limited number of public IP addresses. This process involves translating private IPv4 addresses to public addresses, enabling seamless communication with external networks. Types of NAT There are three
Read More

Splitting Strings in Excel: A Simple Guide

When working with Excel, you may encounter situations where you need to split a string into separate parts. For example, consider the following string: If you want to separate orderId and the actual ID value, there are several methods available depending on your Excel version. Method 1: Using TEXTSPLIT (Excel 365/2021) If you’re using the
Read More

Generating a Calendar Table in Power Query (M Language)

When working with Power BI or other Power Query-supported tools, having a well-structured calendar table is essential for time-based analysis. In this blog post, we will walk through an M Language function that generates a comprehensive calendar table. Why Use a Calendar Table? A calendar table provides essential time-based fields such as year, quarter, month,
Read More

Handling Schema Changes in a Data Warehouse

When building and maintaining a Data Warehouse (DWH), handling schema changes without breaking existing processes is a crucial challenge for data engineers. As new requirements emerge, we often need to add new fields, modify existing structures, or adjust data models while ensuring smooth operation for reporting and analytics. This blog post explores best practices and
Read More

Delta Lake vs. Traditional Data Lakes: Key Differences and Vendor Options

Introduction As data-driven organizations scale their analytics and machine learning workloads, the limitations of traditional data lakes become more apparent. Delta Lake is an open-source storage layer that enhances data lakes with ACID transactions, schema enforcement, and time travel, making them more reliable for big data workloads. In this post, we will explore how Delta
Read More

Why OLTP Systems Don’t Retain Historical Changes

Online Transaction Processing (OLTP) systems are designed for high-speed transactions and efficient data management. However, one of their characteristics is that they do not retain historical changes by default. In this post, we will explore why this happens and provide an example to illustrate the concept. OLTP Systems: Focused on Current Data OLTP databases are
Read More