Correlation measures association
Correlation answers:
- “Do variables move together?”
It does not answer:
- “Does X cause Y?”
Pearson correlation
- Measures linear relationship
- Sensitive to outliers
import numpy as np
from scipy import stats
x = np.array([1, 2, 3, 4, 5, 6])
y = np.array([2, 4, 5, 4, 5, 7])
r, p = stats.pearsonr(x, y)
print("r:", r)
print("p:", p)Spearman correlation
- Uses ranks
- Captures monotonic relationships
- More robust to outliers and non-linearity
import numpy as np
from scipy import stats
x = np.array([1, 2, 3, 4, 5, 6])
y = np.array([10, 9, 7, 6, 3, 1])
rho, p = stats.spearmanr(x, y)
print("rho:", rho)
print("p:", p)Visualize it
Pearson's r measures how tightly two variables move together: near +1 the points hug
a rising line, near 0 there's no pattern, near −1 they hug a falling line. Watch the
cloud tighten and flip as r changes:
Practical guidance
- Plot scatter first.
- Consider transformations (log) if scales vary.
- Be cautious: confounders can create spurious correlation.
Why correlation isn't causation
flowchart LR Z["Confounder
(e.g. summer heat)"] --> X["Ice cream sales"] Z --> Y["Drowning incidents"] X -.->|"correlated, not causal"| Y
Check your understanding
What does Pearson correlation measure?
Pearson measures linear association and is sensitive to outliers. Spearman uses ranks.
Show answer
B — The linear relationship between two variables — Pearson measures linear association and is sensitive to outliers. Spearman uses ranks.
x = 1 to 6 and y = 10, 9, 7, 6, 3, 1 always decreases as x increases. What is Spearman rho?
The relationship is perfectly monotonic and decreasing, so the rank correlation is -1.
Show answer
C — -1 — The relationship is perfectly monotonic and decreasing, so the rank correlation is -1.
When prefer Spearman over Pearson?
Spearman uses ranks, so it is more robust to outliers and non-linearity.
Show answer
B — When you want robustness to outliers and non-linear monotonic patterns — Spearman uses ranks, so it is more robust to outliers and non-linearity.
Two variables have a strong correlation. What does the lesson say about causation?
Nothing in a correlation matrix says which variable, if any, causes the other.
Show answer
B — Confounders can create spurious correlation, so you need domain knowledge or a designed experiment — Nothing in a correlation matrix says which variable, if any, causes the other.
🧪 Try It Yourself
Exercise 1 – Pearson correlation
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 2 – Spearman for a monotonic (non-linear) trend
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 3 – Spotting a possible confounder
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Next
Continue to Non-Parametric Tests for association and comparison tests that don't assume normal, linear relationships.