Date Ranges, Frequencies and Shifting

Why fixed frequencies matter

A real timestamp column (like order_time from the previous page) is often irregular — orders don’t arrive exactly every hour. But for analysis you frequently want a fixed-frequency index: every day, every business day, every 15 minutes. Pandas gives you two tools for this:

• pd.date_range — generate a clean, evenly spaced DatetimeIndex.
• .shift() — move values (or timestamps) forward/backward along that index.

Think of date_range as drawing the ruler, and shift as sliding your data along it.

Generating date ranges with date_range

Basic date_range

import pandas as pd
 
# Start + end (daily by default)
idx = pd.date_range("2024-01-01", "2024-01-06")
print(idx)
 
# Start + number of periods
idx2 = pd.date_range(start="2024-01-01", periods=5)
print(idx2)
 
# Business days only (skips weekends)
idx3 = pd.date_range("2024-01-01", periods=5, freq="B")
print(idx3)

date_range always needs at least two of: start, end, periods. Give it any two and it works out the rest.

Frequency strings and multiples

Frequencies combine a base alias with an optional multiplier:

Alias	Means	Alias	Means
D	Calendar day	ME	Month end
B	Business day	MS	Month start
h	Hour	QE	Quarter end
min	Minute	YE	Year end
s	Second	W-MON	Weekly on Monday

Frequency multiples

import pandas as pd
 
print(pd.date_range("2024-01-01", periods=4, freq="4h"))    # every 4 hours
print(pd.date_range("2024-01-01", periods=4, freq="5min"))  # every 5 minutes
print(pd.date_range("2024-01-01", periods=4, freq="ME"))    # last day of each month

Alias names changed in modern pandas

McKinney’s book uses "M", "T", "H", and "A" for month-end, minute, hour, and year-end. Recent pandas versions renamed these to "ME", "min", "h", and "YE" — the old letters now raise a ValueError telling you exactly which replacement to use. If you copy frequency strings from older tutorials or Stack Overflow answers, expect this to be the first error you hit.

Shifting data: leading and lagging

shift moves the values, keeping the index fixed — this creates NaN at the start or end, which is exactly what you want for lag/lead features:

shift() basics

import pandas as pd
 
ts = pd.Series(
    [100, 105, 102, 108],
    index=pd.date_range("2024-01-31", periods=4, freq="ME"),
)
 
print(ts.shift(1))   # lag by 1 period — value moves down
print(ts.shift(-1))  # lead by 1 period — value moves up

A very common use of shift is computing period-over-period percent change:

Percent change via shift

pct_change = ts / ts.shift(1) - 1
print(pct_change)

If you pass a freq, shift moves the timestamps instead of the values, so no data is discarded:

Shifting the index itself

print(ts.shift(2, freq="ME"))  # every date pushed forward by 2 months

Rolling dates forward with offsets

Pandas offset objects understand calendar rules. Adding MonthEnd() to any date “rolls forward” to the next month end — handy for snapping arbitrary dates onto a frequency grid:

Date offsets

import pandas as pd
from pandas.tseries.offsets import MonthEnd, Day
 
now = pd.Timestamp("2024-01-17")
 
print(now + 3 * Day())        # simple: 3 calendar days later
print(now + MonthEnd())       # rolls forward to 2024-01-31
print(now + MonthEnd(2))      # rolls forward 2 month-ends: 2024-02-29
 
offset = MonthEnd()
print(offset.rollforward(now))  # 2024-01-31
print(offset.rollback(now))     # 2023-12-31 (previous month end)

Common pitfalls

• Naive shift() (no freq) introduces NaN at the start or end — that’s expected, not a bug.
• Legacy aliases "M", "T", "H", "A" are no longer accepted — use "ME", "min", "h", "YE".
• date_range needs exactly two of start / end / periods — passing all three (when inconsistent) or only one raises an error.

Visualize it

diagram From ruler to shifted series mermaid

  flowchart LR
  A["pd.date_range(start, periods, freq)"] --> B["Fixed-frequency DatetimeIndex"]
  B --> C["Series/DataFrame on that index"]
  C --> D["ts.shift(1)
values slide down, NaN at top"]
  C --> E["ts.shift(1, freq='ME')
timestamps slide, no NaN"]

date_range lays down a fixed-frequency ruler; shift then slides data along that ruler, leaving gaps at the edges.

sketch Shifting a timestamp series p5.js Pause Restart

The amber dots are the original values on the ruler; the blue dots show the same values shifted one step to the right.

🧪 Try It Yourself

Exercise 1 – Build a Fixed-Frequency Range

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Lag a Series With shift

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Roll a Date Forward to Month End

pch.exerciseTag pch.exerciseTitle

Next

Fixed-frequency ranges become tricky once you cross regions — see Time Zone Handling for how to localize and convert timestamps safely.