Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql PostHog Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Understanding Distributed Systems whale-alert Yaml

Archives

Category: Programming

Different posts about programming in general

Generating and Uploading Random Data to Azure Blob Storage Using Python

Introduction In today’s data-driven world, automating data generation and storage is crucial for various applications, including testing, data analysis, and machine learning. This blog post will guide you through creating a Python program to generate random data using the Faker library, save it as a CSV file, and upload it to Azure Blob Storage. This
Read More

Decrypting Encrypted Data with Subqueries in SQL

When working with encrypted data in SQL, it’s essential to ensure that the decryption process is secure and efficient. One effective approach is using subqueries. In this post, we’ll demonstrate how to use subqueries to decrypt encrypted data, perform transformations, and filter results based on specific criteria. Scenario Let’s assume we have a table [YOUR_DATABASE].[YOUR_TABLE]
Read More

Extracting the Last Part of a String in SQL Server

Introduction When working with SQL Server, you might often encounter scenarios where you need to extract a specific part of a string. For example, you might have a string in the format DATA_TYPE_001_SAMPLE and you want to extract the last part, SAMPLE. In this blog post, we will explore how to achieve this using SQL
Read More

How to Insert a New Row in a Pandas DataFrame

Working with data often involves modifying it to suit your analysis needs. One common operation is inserting a new row into a DataFrame. In this post, we’ll explore several methods to achieve this in pandas, a powerful data manipulation library in Python. Method 1: Using append() The append() method is straightforward and easy to use
Read More

Loading JSON Data into a pandas DataFrame with Python

In this post, we will walk through the process of loading JSON data into a pandas DataFrame using Python. JSON (JavaScript Object Notation) is a popular data format for exchanging data between a server and a web application. Pandas, a powerful data manipulation library in Python, makes it easy to work with JSON data. Step-by-Step
Read More

Adding SQL Script Filenames to Batch Script Output CSV

When working with batch scripts to execute multiple SQL scripts, it’s often helpful to log not only the results but also the filenames of the executed scripts. This can make it easier to track which script produced which output. In this blog post, we’ll walk through how to modify a batch script to include the
Read More