Matplotlib Data Visualization Tutorial

This document explains the Matplotlib weather data analysis example provided in matplotlib_example.py.

Overview

The example demonstrates how to:

1. Generate simulated weather data
2. Create a DataFrame using pandas
3. Calculate a moving average
4. Create a complex plot using Matplotlib
5. Customize the plot appearance
6. Save and display the plot
7. Calculate and print basic statistics

Code Explanation

Imports and Setup

import os
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from datetime import datetime, timedelta

output_dir = os.path.dirname(os.path.abspath(__file__))

This section imports necessary modules and sets up the output directory for saving the plot.

Data Generation

np.random.seed(42)
start_date = datetime(2023, 1, 1)
dates = [start_date + timedelta(days=i) for i in range(365)]
temperatures = np.random.normal(15, 10, 365) + 5 * np.sin(np.arange(365) * 2 * np.pi / 365)

df = pd.DataFrame({'date': dates, 'temperature': temperatures})

Here, we generate simulated weather data for a full year and create a pandas DataFrame.

Data Processing

df['moving_avg'] = df['temperature'].rolling(window=7).mean()

This calculates a 7-day moving average of temperatures.

Plotting

plt.figure(figsize=(12, 6))
plt.plot(df['date'], df['temperature'], label='Daily Temperature', alpha=0.5)
plt.plot(df['date'], df['moving_avg'], label='7-day Moving Average', color='red')

These lines create the main plot with daily temperatures and the moving average.

Highlighting Seasons

seasons = [
    ('Winter', datetime(2023, 1, 1), datetime(2023, 3, 20)),
    ('Spring', datetime(2023, 3, 21), datetime(2023, 6, 20)),
    ('Summer', datetime(2023, 6, 21), datetime(2023, 9, 22)),
    ('Fall', datetime(2023, 9, 23), datetime(2023, 12, 20)),
    ('Winter', datetime(2023, 12, 21), datetime(2023, 12, 31))
]

for season, start, end in seasons:
    plt.axvspan(start, end, alpha=0.2, label=season if season not in plt.gca().get_legend_handles_labels()[1] else "")

This section highlights different seasons on the plot using axvspan.

Plot Customization

plt.title('Daily Temperatures and 7-day Moving Average (2023)')
plt.xlabel('Date')
plt.ylabel('Temperature (°C)')
plt.legend()
plt.grid(True, linestyle='--', alpha=0.7)
plt.gcf().autofmt_xdate()

These lines customize various aspects of the plot, including title, labels, legend, grid, and date formatting.

Saving and Displaying the Plot

plt.tight_layout()
output_file_path = os.path.join(output_dir, 'temperature_analysis.png')
plt.savefig(output_file_path, dpi=300)
plt.show()

This saves the plot as a high-resolution PNG file and displays it.

Statistical Analysis

hottest_day = df.loc[df['temperature'].idxmax()]
coldest_day = df.loc[df['temperature'].idxmin()]

print(f"Hottest day: {hottest_day['date'].strftime('%Y-%m-%d')} with {hottest_day['temperature']:.2f}°C")
print(f"Coldest day: {coldest_day['date'].strftime('%Y-%m-%d')} with {coldest_day['temperature']:.2f}°C")
print(f"Average temperature: {df['temperature'].mean():.2f}°C")

This section calculates and prints basic statistics about the temperature data.

Here's a detailed example showcasing key Matplotlib features:

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from datetime import datetime, timedelta

# Simulate weather data
np.random.seed(42)
start_date = datetime(2023, 1, 1)
dates = [start_date + timedelta(days=i) for i in range(365)]
temperatures = np.random.normal(15, 10, 365) + 5 * np.sin(np.arange(365) * 2 * np.pi / 365)

# Create DataFrame
df = pd.DataFrame({'date': dates, 'temperature': temperatures})

# Calculate moving average
df['moving_avg'] = df['temperature'].rolling(window=7).mean()

# Create the plot
plt.figure(figsize=(12, 6))

# Plot daily temperatures
plt.plot(df['date'], df['temperature'], label='Daily Temperature', alpha=0.5)

# Plot moving average
plt.plot(df['date'], df['moving_avg'], label='7-day Moving Average', color='red')

# Highlight seasons
seasons = [
    ('Winter', datetime(2023, 1, 1), datetime(2023, 3, 20)),
    ('Spring', datetime(2023, 3, 21), datetime(2023, 6, 20)),
    ('Summer', datetime(2023, 6, 21), datetime(2023, 9, 22)),
    ('Fall', datetime(2023, 9, 23), datetime(2023, 12, 20)),
    ('Winter', datetime(2023, 12, 21), datetime(2023, 12, 31))
]

for season, start, end in seasons:
    plt.axvspan(start, end, alpha=0.2, label=season if season not in plt.gca().get_legend_handles_labels()[1] else "")

# Customize the plot
plt.title('Daily Temperatures and 7-day Moving Average (2023)')
plt.xlabel('Date')
plt.ylabel('Temperature (°C)')
plt.legend()
plt.grid(True, linestyle='--', alpha=0.7)

# Rotate and align the tick labels so they look better
plt.gcf().autofmt_xdate()

# Save the plot
plt.tight_layout()
plt.savefig('/path/to/your/output/temperature_analysis.png', dpi=300)

# Display the plot
plt.show()

# Calculate and print some statistics
hottest_day = df.loc[df['temperature'].idxmax()]
coldest_day = df.loc[df['temperature'].idxmin()]

print(f"Hottest day: {hottest_day['date'].strftime('%Y-%m-%d')} with {hottest_day['temperature']:.2f}°C")
print(f"Coldest day: {coldest_day['date'].strftime('%Y-%m-%d')} with {coldest_day['temperature']:.2f}°C")
print(f"Average temperature: {df['temperature'].mean():.2f}°C")

Break Down of The Key Features Demonstrated in This Example

1. Data Generation and Manipulation:

   • We simulate a year's worth of daily temperature data using NumPy and Pandas.
   • This showcases how data engineers often need to work with time series data and perform date-based operations.

2. Moving Average Calculation:

   • We calculate a 7-day moving average using Pandas' rolling() function.
   • Moving averages are crucial for smoothing out short-term fluctuations and highlighting longer-term trends in time series data.

3. Multiple Line Plots:

   • We plot both daily temperatures and the moving average on the same graph.
   • This demonstrates how to compare raw data with processed data, a common task in data analysis.

4. Customizing Plot Appearance:

   • We use various Matplotlib functions to customize the plot's appearance, including colors, labels, title, and grid.
   • Proper visualization is key in data engineering for presenting insights effectively.

5. Highlighting Seasons:

   • We use plt.axvspan() to highlight different seasons on the plot.
   • This shows how to add context to data visualizations, which is important for making data more interpretable.

6. Date Handling:

   • We work with datetime objects and format dates for the x-axis.
   • Proper date handling is crucial in many data engineering tasks, especially when dealing with time series data.

7. Saving and Displaying Plots:

   • We demonstrate how to save the plot as a high-resolution image and display it.
   • This is important for including visualizations in reports or dashboards.

8. Basic Statistical Analysis:

   • We calculate and print some basic statistics about the data.
   • Data engineers often need to provide quick summaries of data characteristics.

This example illustrates how Matplotlib can be used in a data engineering workflow to visualize and analyze time series data. It demonstrates the library's flexibility in creating custom visualizations that combine multiple data series, highlight specific periods, and present data in a clear, informative manner. These skills are essential for data engineers who need to explore data patterns, validate data processing steps, and communicate insights effectively to stakeholders.

Running the Example

To run this example:

1. Ensure you have Matplotlib, NumPy, and pandas installed:

   pip install matplotlib numpy pandas
   

2. Run the script:

   python matplotlib_example.py
   

The script will generate a plot of simulated temperature data, save it as 'temperature_analysis.png' in the same directory as the script, display the plot, and print some basic statistics about the data.