Non-Parametric Tests (Mann-Whitney, Wilcoxon)

Why non-parametric tests

When data is:

• Not normally distributed
• Heavy-tailed
• Contains outliers

…it can be safer to compare median/rank behavior.

Mann–Whitney U test (two independent groups)

Alternative to the independent t-test.

Mann–Whitney

import numpy as np
from scipy import stats
 
A = np.array([1, 2, 2, 3, 100])
B = np.array([1, 1, 2, 2, 3])
 
u, p = stats.mannwhitneyu(A, B, alternative="two-sided")
print("U:", u)
print("p:", p)

Wilcoxon signed-rank (paired)

Alternative to the paired t-test.

Wilcoxon

import numpy as np
from scipy import stats
 
before = np.array([10, 12, 11, 9, 13])
after  = np.array([11, 12, 12, 10, 14])
 
w, p = stats.wilcoxon(after - before)
print("W:", w)
print("p:", p)

Notes

• Non-parametric tests often have less power when assumptions for parametric tests are satisfied.
• Always combine tests with visualizations.

Choosing parametric vs. non-parametric

diagram When to switch to a non-parametric test mermaid

  flowchart TD
  A["Comparing two groups"] --> B["Data roughly normal,
no big outliers?"]
  B -->|"Yes"| C["t-test
(parametric)"]
  B -->|"No"| D["Same subjects twice?"]
  D -->|"Yes"| E["Wilcoxon
signed-rank"]
  D -->|"No"| F["Mann-Whitney U"]

Outliers, heavy skew, or a broken normality assumption point toward rank-based tests.

From the book

Rank-based tests trade a little statistical power for a lot of robustness — the same trade-off McKinney flags when recommending median()/quantile() over mean() for skewed pandas columns. If your data has the kind of long tail typical of revenue or latency metrics, reach for Mann-Whitney or Wilcoxon before assuming a t-test is safe.

🧪 Try It Yourself

Exercise 1 – Mann-Whitney U test

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Wilcoxon signed-rank test

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Deciding which test fits

pch.exerciseTag pch.exerciseTitle

Next

Continue to A/B Testing Basics to apply these comparison tests to a real product experiment.