Geek Logbook

Tech sea log book

Month: October 2022

Enviroments in Virtual Env

The importance of using environments As was said in Setting environments in Python it’s important to use environments for your deployment, even if these are side projects or wild repositories. But at first, I used Conda to create the environment. However, a while ago, I made the switch to Virtual Env As a point of
Read More

Event Driven Architectures

In the Gartner Submit of 2006 Mani Chandy talked about the existence of a misconception of Event Driven Architecture (EDA). So, he proposed to talk about the understanding of EDA and its Return of Investment. So he can evaluate the true costs and benefits. In this talk an event was defined as:  “An event is
Read More

Matei Zaharia – Spark: The Definitive Guide. Common Operations

Define Schemas manually When using Spark for production Extract, Transform, and Load (ETL), it is often a good idea to define your schemas manually, especially when working with untyped data sources like CSV and JSON, because schema inference can vary depending on the type of data that you read in. (Chambers, 2017, 66) SQL Expressions
Read More

Kleppmann – Designing Data Intensive Applications

A data-intensive application is typically built from standard building blocks that provide commonly needed functionality. For example, many applications need to: • Store data so that they, or another application, can find it again later (databases) • Remember the result of an expensive operation, to speed up reads (caches) • Allow users to search data
Read More

Testing in Python: Pytest Vs Unit test

How important are the tests? Testing is one of the most important skills we need to develop once we join the industry. In fact, knowing about testing is something that is not as evaluated as it could be. In general, the challenges are related to having logic related to programming skills or knowing about design
Read More