Geek Logbook

Tech sea log book

Recent Posts

Categories

#50projectsIn50days Airflow Amazon Web Services Apache Iceberg AWS Azure bash bastion host Batch CAP Coders At Work Data data lakes Data Warehouse Docker Edsger W. Dijkstra Excel Git Hadoop HDFS Hellerstein Hive Javascript Networking NoSql Power Bi Powershell Python Redash S3 Social Skills Software Developer Spark Spark: The Definitive Guide SQL SSH terraform Testing The Elements of Programming Style Traversy Media Ubuntu Understanding Distributed Systems Virtual Box whale-alert Yaml

Archives

Tag: Spark

Understanding .master() in Apache Spark

In Apache Spark, the .master() method is used to specify how your application will run, either on your local machine or on a cluster. Choosing the correct option is essential depending on your environment. This post will explain the different .master() options in Spark and when to use them. Local Mode The local mode runs
Read More

Running PySpark on Google Colab: Do You Still Need findspark?

Introduction For a long time, using Apache Spark in Google Colab required manual setup, including installing Spark and configuring Python to recognize it. This was often done using the findspark library. However, recent changes in Colab have made this process much simpler. In this post, we will explore whether findspark is still necessary and the
Read More