t-test (independent and paired)

When to use t-tests

Use t-tests when you want to compare means.

• Independent (two-sample): two different groups
• Paired: before/after measurements on the same subjects

Independent two-sample t-test

Example: average spend differs between two groups.

Independent t-test

import numpy as np
from scipy import stats
 
A = np.array([120, 140, 130, 150, 160])
B = np.array([100, 90, 110, 95, 105])
 
t_stat, p = stats.ttest_ind(A, B, equal_var=False)  # Welch t-test
print("t:", t_stat)
print("p:", p)

Welch vs pooled variance

• Welch (equal_var=False) is safer when variances differ.
• Pooled assumes equal variances.

Paired t-test

Example: same users before/after a change.

Paired t-test

import numpy as np
from scipy import stats
 
before = np.array([10, 12, 11, 9, 13])
after  = np.array([11, 12, 12, 10, 14])
 
t_stat, p = stats.ttest_rel(after, before)
print("t:", t_stat)
print("p:", p)

Assumptions (important)

• Data points are independent (except paired case)
• Approximately normal within each group (CLT helps with larger n)
• No extreme outliers

If assumptions are shaky, consider non-parametric alternatives.

Which t-test do you need?

diagram Choosing between t-tests mermaid

  flowchart TD
  A["Comparing two means?"] --> B["Same subjects,
measured twice?"]
  B -->|"Yes"| C["Paired t-test
(ttest_rel)"]
  B -->|"No"| D["Independent t-test
(ttest_ind)"]
  D --> E["Equal variances?"]
  E -->|"Unsure/No"| F["Welch t-test
(equal_var=False)"]
  E -->|"Yes"| G["Pooled t-test
(equal_var=True)"]

The decision hinges on whether the two groups share the same subjects.

From the book

statsmodels’ formula API (smf.ols("y ~ group", data=df)) can reproduce a two-sample t-test as a linear regression with one categorical predictor — the t-statistic on that group coefficient matches stats.ttest_ind. It’s a useful bridge once you move from standalone tests toward the modeling libraries covered later.

🧪 Try It Yourself

Exercise 1 – Independent (Welch) t-test

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Paired t-test (before/after)

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Picking the right test

pch.exerciseTag pch.exerciseTitle

Next

Continue to ANOVA (one-way) to compare means across three or more groups at once.