Geek Logbook

Tech sea log book

Running Apache Airflow Across Environments

Apache Airflow has become a de facto standard for orchestrating data workflows. However, depending on the environment, the way Airflow runs can change significantly. Many teams get confused when moving between managed cloud services, local setups, and containerized deployments. This post provides a clear comparison of how Airflow operates in different contexts: 1. Airflow on

Testing Apache Airflow DAGs: A Modular Approach

Introduction Apache Airflow is a powerful workflow automation tool, but testing DAGs can be challenging due to their dependency on the Airflow scheduler and execution environment. In this post, we explore a modular approach that enables easier testing by extracting logic into separate Python functions, which can be tested independently. Methods for Testing Airflow DAGs