Hierarchical Indexing (MultiIndex)

Why stack more than one index level

Sometimes one label isn’t enough to describe a row — “city” alone doesn’t tell you which year; “year” alone doesn’t tell you which city. A MultiIndex lets a single axis carry two or more levels at once, so you can keep working with a Series or DataFrame instead of juggling nested dictionaries.

A Series with a MultiIndex

import pandas as pd
import numpy as np
 
data = pd.Series(
    np.arange(9),
    index=[
        ["a", "a", "a", "b", "b", "c", "c", "d", "d"],
        [1, 2, 3, 1, 3, 1, 2, 2, 3],
    ],
)
print(data)

The blank-looking gaps in the printed output mean “same as the label directly above” — it’s just a display convenience, the index itself has no gaps.

Partial indexing

With a hierarchical index, you can select using just the outer level:

Partial indexing on the outer level

print(data["b"])         # every row where the outer level is "b"
print(data.loc["b":"c"]) # a slice across outer-level labels

You can also reach into the inner level directly:

Selecting from the inner level

print(data.loc[:, 2])  # every row where the inner level equals 2

Reshaping with unstack

A hierarchically indexed Series can be pivoted into a DataFrame with unstack — the inner level becomes columns:

unstack turns levels into columns

print(data.unstack())

Missing combinations (like "b" + 2, which never existed) become NaN.

Building a MultiIndex from DataFrame columns: set_index

More often, you’ll have plain columns that you want to promote into a hierarchical row index:

set_index builds a MultiIndex

frame = pd.DataFrame({
    "city": ["pune", "pune", "delhi", "delhi"],
    "year": [2023, 2024, 2023, 2024],
    "sales": [100, 120, 200, 220],
})
 
frame2 = frame.set_index(["city", "year"])
print(frame2)

reset_index does the reverse — it moves the index levels back into ordinary columns:

reset_index undoes set_index

print(frame2.reset_index())

Reordering and sorting levels

swaplevel and sort_index(level=...)

print(frame2.swaplevel("city", "year"))       # levels interchanged, data untouched
print(frame2.sort_index(level="year"))        # sort using just one level

Aggregating by level

Many summary methods accept a level argument, so you can aggregate across just one part of the index without unstacking first:

groupby(level=...) for level-wise stats

print(frame2.groupby(level="city").sum())

Common pitfalls

• Partial indexing performance is much better when the index is sorted starting from the outermost level — call sort_index() after building a MultiIndex if you’ll be slicing it a lot.
• set_index drops the columns you passed by default; pass drop=False if you still want them available as regular columns too.
• Selecting an inner level directly (data.loc[:, 2]) uses a different syntax than selecting the outer level (data["b"]) — easy to mix up at first.

Visualize it

diagram From flat columns to a MultiIndex tree mermaid

  flowchart LR
  A["Flat columns:
city, year, sales"] -->|"set_index(['city','year'])"| B["MultiIndex rows:
city -> year -> sales"]
  B -->|"reset_index()"| A
  B -->|"groupby(level='city')"| C["Aggregated by outer level"]

set_index promotes ordinary columns into hierarchical row-index levels; reset_index moves them back to being plain columns.

sketch A MultiIndex as a nested tree p5.js Pause Restart

Each city branches into its years; sales values live at the leaves — this is what a two-level MultiIndex looks like structurally.

From the book

McKinney frames hierarchical indexing as “a way for you to work with higher dimensional data in a lower dimensional form” — a two-level MultiIndex Series behaves like a small 2D table without needing a full DataFrame. This is exactly why groupby on a duplicate-timestamp Series (covered earlier in this phase) and set_index on multiple columns both lean on the same underlying MultiIndex machinery.

🧪 Try It Yourself

Exercise 1 – Select From the Outer Level

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Build a MultiIndex With set_index

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Aggregate by One Index Level

pch.exerciseTag pch.exerciseTitle

Next

A MultiIndex is one way to reshape data into a different structure — continue to Reshaping Data (pivot, pivot_table, melt) for more ways to move between wide and long formats.