Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql PostHog Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Understanding Distributed Systems whale-alert Yaml

Archives

Category: Notes

Notes taking about the different books I’ve read since the blog started. Usually related to specific technology but, sometimes I read about tech history.

Understanding the Differences Between Parquet, Avro, JSON, and CSV

When working with data, choosing the right file format can significantly impact performance, storage efficiency, and ease of use. In this post, we will compare four widely used data formats: Parquet, Avro, JSON, and CSV. Each has its strengths and weaknesses, making them suitable for different scenarios. 1. Parquet Overview: Parquet is a columnar storage
Read More

Understanding the CAP Theorem in NoSQL Databases

The CAP theorem (Consistency, Availability, and Partition Tolerance) plays a crucial role in designing and selecting NoSQL databases. This theorem states that in a distributed system, it is impossible to achieve all three properties simultaneously: How CAP Theorem Relates to NoSQL Databases NoSQL databases are designed for scalability and flexibility, often trading off one CAP
Read More

Understanding Docker Engine Components

Docker Engine is an open-source platform that has revolutionized how applications are developed, deployed, and executed using container technology. By encapsulating applications and their dependencies in lightweight, portable containers, Docker ensures consistent behavior across different environments. Understanding the fundamental components of Docker is crucial to fully leveraging its capabilities. Core Components of Docker Engine 1.
Read More

What Does an Exploratory Data Analysis (EDA) Evaluate?

An Exploratory Data Analysis (EDA) is a critical step in the data analysis process that focuses on evaluating and examining data to uncover its main characteristics. It is performed before delving deeper into analysis or building predictive models. The primary purpose of an EDA is to understand the dataset, identify issues, and gain insights that
Read More

Exploring Free Resources to Learn AWS and Azure Cloud Platforms

Cloud computing is an essential skill in today’s tech landscape. Among the major players, AWS and Azure stand out as leading cloud platforms, offering a wealth of free resources to help individuals learn and experiment. This blog post outlines some of the most valuable free tools, learning paths, and tips for getting started with AWS
Read More

Root Cause Analysis (RCA) for Data

Introduction In the realm of data management and analysis, problems can range from data quality issues to processing errors and performance bottlenecks. Identifying the root cause of these issues is crucial for ensuring data integrity and reliability. Root Cause Analysis (RCA) is a systematic approach to uncovering the underlying causes of data problems and implementing
Read More

Understanding Data Layout, Files, and Tree Indexes: An Overview

In this post, we’ll explore several fundamental concepts related to data storage and indexing: Data Layout, Files, Tree Indexes, and B+ Trees. Understanding these concepts is crucial for anyone working with databases or file systems. Data Layout Data layout refers to how data is physically arranged on storage devices. This includes: Proper data layout is
Read More

Effective Knowledge Transfer of Data: Key Elements

Transferring knowledge, especially when it involves data, is a critical proces. Especially between consultants. Whether you’re transitioning to a new team, it’s crucial to get this process right. Here are five essential elements to ensure effective knowledge transfer of data: 1. Comprehensive and Clear Documentation This involves providing detailed information about the data’s origin, its
Read More

Minimizing Operational Overhead of EC2 Fleet OS Security Governance in AWS: Recommendations for DevOps Teams

Minimizing the operational overhead of EC2 fleet OS security governance is essential for maintaining a secure and efficient AWS environment. In this blog post, we’ll explore the challenges faced by DevOps teams in managing EC2 fleet OS security and provide recommendations to minimize operational overhead. Challenges in EC2 Fleet OS Security Governance Managing the security
Read More