Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql PostHog Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Understanding Distributed Systems whale-alert Yaml

Archives

Tag: AWS

How to Disable an AWS Glue Trigger from the CLI

When working with AWS Glue, triggers are an important mechanism to orchestrate jobs or workflows. Sometimes, however, you may need to temporarily disable a trigger without deleting it—for example, to pause scheduled ingestions during maintenance or testing. This article explains how to disable a trigger using the AWS CLI. Understanding AWS Glue Triggers AWS Glue
Read More

Reducing AWS Costs: How to Temporarily Stop an Aurora Serverless v2 Cluster

When managing cloud infrastructure, minimizing costs without compromising data integrity is a continuous priority. Amazon Aurora Serverless v2 offers scalability and high availability, but unlike traditional RDS instances, it introduces nuances in how compute resources are billed. One common question arises: Can an Aurora Serverless v2 database be stopped to save costs? Understanding Aurora Serverless
Read More

EMR vs AWS Glue: Choosing the Right Data Processing Tool on AWS

When working with big data on AWS, two commonly used services for data processing are Amazon EMR and AWS Glue. Although both support scalable data transformation and analytics, they differ significantly in architecture, control, use cases, and cost models. Choosing the right tool depends on your specific workload, performance needs, and operational preferences. In this
Read More

The History and Evolution of Amazon S3: Was It Ever Based on HDFS?

When discussing cloud storage today, Amazon S3 is almost synonymous with scalable, reliable object storage. However, a common question among those familiar with big data technologies like Hadoop is:Was Amazon S3 ever based on HDFS (Hadoop Distributed File System)? The short answer is: No. Amazon S3: Launched Before HDFS Amazon S3 was officially launched on
Read More

Understanding Subnets, Gateways, and Route Tables in AWS

When designing applications in AWS, it’s crucial to understand how networking components interact within a Virtual Private Cloud (VPC). This post will cover subnets, gateways, and route tables, specifically in the context of the AWS Free Tier. Everything Lives Inside a VPC A VPC (Virtual Private Cloud) is a logically isolated section of AWS where
Read More

Understanding Stateful vs. Stateless Firewalls in AWS

When working with network security, it’s crucial to understand the difference between stateful and stateless firewalls. In AWS, this understanding is particularly important when configuring security groups and network access control lists (ACLs). Stateless Firewalls A stateless firewall handles each packet independently, without remembering any information about previous packets. This means that you must explicitly
Read More

Minimizing Operational Overhead of EC2 Fleet OS Security Governance in AWS: Recommendations for DevOps Teams

Minimizing the operational overhead of EC2 fleet OS security governance is essential for maintaining a secure and efficient AWS environment. In this blog post, we’ll explore the challenges faced by DevOps teams in managing EC2 fleet OS security and provide recommendations to minimize operational overhead. Challenges in EC2 Fleet OS Security Governance Managing the security
Read More

Implementing Resilient Architectures in AWS: Strategies for Automated Recovery and Testing

Implementing resilient architectures in AWS is essential for ensuring high availability and reliability of your applications. In this blog post, we’ll explore strategies for automating recovery and testing to improve the resilience of your AWS environment. Monitoring for Key Performance Indicators (KPIs) Monitoring your workload for key performance indicators (KPIs) is essential for detecting and
Read More

Enabling Traceability and Auditing Security Events in AWS: Best Practices and Tools

Traceability and auditing of security events are crucial for maintaining the security and compliance of your AWS environment. In this blog post, we’ll explore how to enable traceability and auditing in AWS, including monitoring, alerting, and auditing actions and changes in your environment. Importance of Traceability and Auditing Traceability and auditing allow you to: Tools
Read More