Correlation and Covariance

What correlation and covariance answer

Both measure how two variables move together:

• Covariance tells you the direction of the relationship (positive or negative) but its size depends on the scale of your data, which makes it hard to compare across different variables.
• Correlation rescales covariance into a number between -1 and 1, so it's directly comparable no matter what units the original data was in.

A correlation near 1 means "moves up together," near -1 means "one goes up as the other goes down," and near 0 means "no clear linear relationship."

Series.corr and Series.cov

import pandas as pd
 
ad_spend = pd.Series([1, 2, 3, 4, 5])
sales = pd.Series([2, 4, 6, 8, 10])  # perfectly proportional to ad_spend
 
print(round(ad_spend.corr(sales), 2))  # 1.0 — perfectly correlated
print(ad_spend.cov(sales))             # 5.0 — positive, but scale-dependent

Both methods automatically use only the overlapping, non-missing, aligned-by-index values from the two Series — you don't need to clean them up first.

DataFrame.corr and DataFrame.cov: full matrices

Call .corr() (or .cov()) directly on a DataFrame to get every pairwise combination of columns at once:

df = pd.DataFrame({
    "x": [1, 2, 3, 4, 5],
    "y": [2, 4, 6, 8, 10],
    "z": [5, 3, 4, 2, 1],
})
 
print(df.corr())

The diagonal is always 1.0 — every column is perfectly correlated with itself. z here trends opposite to x and y, so its correlation with them is negative.

corrwith: one column against many

corrwith computes the correlation of every column in a DataFrame against a single Series (or another DataFrame's matching columns) — useful for quickly checking which columns move with a target metric:

print(df.corrwith(df["x"]))

Common pitfalls

• Correlation measures a linear relationship only — two variables can have a strong non-linear pattern (like a U-shape) and still show a correlation near 0.
• A high correlation is not causation — McKinney's book pairs this topic with data alignment precisely because spurious correlations often come from misaligned data.
• corrwith(axis="columns") compares row-by-row instead of column-by-column — easy to reach for the wrong axis by habit.

Visualize it

  flowchart LR
  A["Series A, Series B
(aligned by index)"] --> B["cov(A, B)
direction, but scale-dependent"]
  B --> C["corr(A, B)
rescaled to -1 .. 1"]
  D["Whole DataFrame"] --> E["df.corr()
matrix of every pair"]

🧪 Try It Yourself

Exercise 1 – Compute Correlation Between Two Series

Press Run to see what your code prints.

Exercise 2 – Build a Full Correlation Matrix

Press Run to see what your code prints.

Exercise 3 – Compare Many Columns Against One Target

Press Run to see what your code prints.

Check your understanding

Next

Correlation compares columns side by side. Grouping and Aggregations (groupby, agg) covers summarizing rows instead — split, apply, and combine.