Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Traversy Media Understanding Distributed Systems whale-alert Yaml

Archives

Understanding client_ingestion_warning in PostHog: Are You Losing Data?

When using PostHog with the default posthog-js configuration, you may encounter the following warning: In the UI, this appears as: This article explains what it means, why it happens, and whether you are losing data. What Is client_ingestion_warning? client_ingestion_warning is an internal warning emitted by the PostHog JavaScript SDK when the client-side rate limiter is
Read More

Hardening OAuth Token Management in Postman: Preventing Environment Cross-Contamination

When working with multiple third-party APIs (Zoom, HubSpot, Meta, etc.), a common operational risk in Postman is environment cross-contamination. Tokens may be overwritten unintentionally if the wrong environment is active. This article describes a controlled, production-grade approach to managing OAuth tokens safely in Postman. The Core Problem If all environments share a variable named: and
Read More

Daily Failure Reporting in DynamoDB Using Lambda, EventBridge Scheduler, and SES

Operational monitoring requires structured visibility into failures. If your processes write execution logs to DynamoDB and mark failed executions with status = FAILED, you can implement a deterministic daily reporting pipeline using AWS Lambda, EventBridge Scheduler, and Amazon SES. This article describes a single, production-grade implementation. Objective Architecture This solution is fully serverless and horizontally
Read More

Understanding ip-api Batch Limits and Effective Throughput

When integrating IP geolocation into a data pipeline, understanding rate limits and batching constraints is essential. This post analyzes the practical limits of the ip-api free tier and how to compute effective throughput. 1. Free Tier Constraints The ip-api free plan imposes the following restrictions: These limits apply globally per source IP address. 2. Maximum
Read More

Window Functions vs JOIN in Spark: A Physical Plan Perspective

When solving analytical queries in Spark SQL, there are often multiple correct formulations. However, they do not produce equivalent execution plans. This article compares two approaches to the same problem: “Find the second highest salary per department, but only in departments with at least two employees.” We analyze which approach is more efficient and why,
Read More

Can You Know the Location of an IPv6 Address?

Example IPv6: Short answer: only approximately, and with significant limitations. This article explains what can and cannot be inferred from an IPv6 address, the technical reasons behind those limitations, and how geolocation services actually work. 1. IPv6 Structure and Why It Matters An IPv6 address is 128 bits long and typically structured as: In practice:
Read More

AWS Glue + Chargebee: Diagnosing CERTIFICATE_VERIFY_FAILED After TLS Chain Updates

Context An AWS Glue job that consumes the Chargebee API begins failing with: The same request works in Postman. This pattern typically appears after a certificate chain rotation on the API provider side combined with an outdated trust store in the execution environment. Chargebee announced updates related to its TLS certificate chain (DigiCert G2 becoming
Read More

Why There Is No “Interpreter” Endpoint in the Zoom API

Many teams attempt to retrieve language interpretation usage (e.g., minutes consumed per language channel) through the Zoom REST API, only to discover that no such endpoint exists for Meetings or Webinars. This is not a misconfiguration or missing scope issue — it is a structural limitation of the public API. This article explains: The Common
Read More

Why You Can’t Get Full Social Analytics from the HubSpot API (Even with Marketing Hub Pro)

Many teams assume that upgrading to Marketing Hub Professional unlocks full programmatic access to social media performance metrics. It does not. This article clarifies what is technically possible, what is not, and how to architect a reliable data pipeline for social analytics. The Core Limitation HubSpot allows you to: But HubSpot does not provide an
Read More

From OLTP to OLAP: How Data Moves from 3NF to a Dimensional Data Warehouse

Modern data architectures typically separate operational systems from analytical systems. This separation is not accidental—it reflects fundamentally different workloads, data models, and optimization strategies. This article explains the conceptual transition: Operational Systems (OLTP) and 3rd Normal Form Transactional systems—CRM platforms, payment processors, ERPs, application databases—are designed for: These systems are usually modeled in Third Normal
Read More