Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash Batch CAP Coders At Work Data data lakes Data Pipelines Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql PostHog Postman Power Bi Powershell Pyspark Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Understanding Distributed Systems whale-alert Yaml

Archives

What is programming?

‘ve finished reading “Coders at work” a series of interviews between @peterseibel and well-known programmers/coders/(etc). The first edition was in 2009. And in the preface you can read: Yet despite the millions of people who have written code, and the billions, if not trillions of lines of code written since the field began, it still
Read More

Python calculate seconds and total_seconds

If you want to calculate the total seconds between two dates. You could be tempted to do a time delta and see the seconds. But this approach will give you an unexpected result. You have to use time_second. Return the total number of seconds contained in the duration.  https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds Source:

What is an ‘Ephemeral cluster’?

When you create a service to compute, for example in HD-Insight you can create a cluster which remains active once it’s created or, in the other hand, stop (will be ‘deleted’) after some amount of innactivities. If you, like me, your first approach was with the Azure Service you’ll notice that the name if the
Read More

Peter Norvig Paper: Oh shinny! antidote

Dark Knights In the TED talks The mind behind Linux | Linus Torvalds One of the comments that Linus Said was: Edison may not have been a nice person, he did a lot of things — he was maybe not so intellectual, not so visionary. But I think I’m more of an Edison than a Tesla. Linus Torvals So
Read More

Logger In python – First Approach

Besides using the print statements and the debugging tools sometimes (more and more frequently) I’m seeing in the code the logging module. According to the python documentation: This module defines functions and classes which implement a flexible event logging system for applications and libraries. The key benefit of having the logging API provided by a
Read More

Set secrets in Databricks

If you add as plain text the user and password of your connections you are making a mistake that it’s easy to solve. In order to solve Now you can use the CLI and it should be connected to your instances. So, now we can run the commands to set the keys. For example, if
Read More

Difference between Framework and Libraries

Software Development has tricky words. Some Jargon that seems as unreachable when we are starting. Even though is not a game changer understand this difference is a nice to have and in one or two paragraph I’m going to point the ideas that give me some impression that what are the key points in the
Read More

Resource from Vanguard ETF

Looking for vanguard ETF data is not an easy task. Because there are a lot of pages that need to subscribe or even purchase a subscription. So, we can’t access to free information. I don’t know if this behavior is normal across the industry, or I am thinking wrong about that. Because the composition of
Read More

Load data from Snowflake to S3

If you want to load data from Snowflake to S3 should try to use the COPY INTO command so, you run something like this command in the snowflake Web App: copy into @my_ext_unload_stage/d1 from mytable; And receive the following error: Failure using stage area. Cause: [Access Denied (Status Code: 403; Error Code: AccessDenied)] To solve
Read More