Logging is a vital part of any application, offering insights into the application’s flow, performance, and error handling. In many scenarios, you may want to log messages both to the console and a file, but with different levels of detail. For example, you might display INFO and WARNING messages in the console while logging more detailed DEBUG messages in a file.

In this post, we’ll demonstrate how to achieve this using the logging module, applying it to a real-world example where we retrieve stock prices using yfinance.

Requirements

Before we proceed, ensure you have installed the necessary packages:

pip install yfinance

The Objective

Our goals:

• Log DEBUG and higher-level messages (including errors and warnings) into a file.
• Log only INFO and WARNING messages to the console.
• Display stock prices for companies below a specified price threshold.

The Code

Here’s a streamlined version of the Python script:

import yfinance as yf
import json
import logging

# Configure logging
logger = logging.getLogger("tickers_below_five")
logger.setLevel(logging.DEBUG)  # Capture all messages

# File handler for logging DEBUG and above messages
file_handler = logging.FileHandler('logs\\below_five.log', encoding='utf-8')
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))

# Console handler for logging INFO and above messages
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_handler.setFormatter(logging.Formatter('%(levelname)s - %(message)s'))

# Adding handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(console_handler)

# Main logic
json_file_path = "sources\\company_tickers.json"
price_threshold = 5
low_price_stocks = []

try:
    with open(json_file_path, 'r') as j:
        contents = json.loads(j.read())

        for key, value in contents.items():
            cik_str = value.get('cik_str')
            ticker = value.get('ticker')
            title = value.get('title')

            try:
                stock = yf.Ticker(ticker)
                hist = stock.history(period="1d")
                
                if hist.empty:
                    logger.warning(f"Key:{key}, Ticker: {ticker}, Title: {title}, Price: Data not available")
                    continue
                
                price = hist['Close'].values[0]
                logger.info(f"Key:{key}, Ticker: {ticker}, Title: {title}, Price: {price}")

                if price < price_threshold:
                    low_price_stocks.append((title, ticker, price))

            except Exception as e:
                logger.error(f"Key:{key}, Ticker: {ticker}, Title: {title}, Error: {e}")

except Exception as e:
    logger.error(f"Error reading JSON file: {e}")

# Final log output
logger.info(f"\nLow price stocks: {low_price_stocks}")

How It Works

1. Logging Configuration:
   • We set up two handlers:
     • File handler logs all messages (DEBUG level and above) to a log file named below_five.log.
     • Console handler shows only INFO and WARNING messages to keep the console output clean.
   The configuration makes it easy to track detailed logs in a file for debugging while keeping the console less cluttered with just important messages.
2. Fetching Stock Prices:
   • The script opens a JSON file containing company tickers.
   • For each company, it uses the yfinance library to fetch stock data for the past day and logs the stock price, warnings if data is unavailable, and errors if an exception occurs.
3. Low-Price Stocks:
   • Stocks that fall below the specified price_threshold (in this case, 5) are logged and displayed.

Console Output

In the console, you’ll only see the INFO and WARNING messages, which are most relevant to the user:

INFO - Key:123, Ticker: XYZ, Title: Example, Price: 4.75
WARNING - Key:456, Ticker: ABC, Title: Another, Price: Data not available
INFO - 
Low price stocks: [('Example', 'XYZ', 4.75)]

Log File Output

The log file (below_five.log) captures all the DEBUG messages and above, including ERROR and INFO logs with detailed timestamps:

2024-09-29 12:34:56,789 - INFO - Key:123, Ticker: XYZ, Title: Example, Price: 4.75
2024-09-29 12:34:57,000 - WARNING - Key:456, Ticker: ABC, Title: Another, Price: Data not available
2024-09-29 12:34:58,123 - ERROR - Key:789, Ticker: DEF, Title: Another Example, Error: Data fetch failed

Conclusion

By separating log output into different handlers, this script allows you to effectively track important information in the console while maintaining a more detailed log for debugging. This method is ideal for applications where you want concise console feedback but retain deeper insights in a log file.

Feel free to adapt this logging setup for your own projects, whether you’re building stock data pipelines or handling other forms of data processing!