Skip to content
Python Central Hub

Date Ranges, Frequencies and Shifting

Why fixed frequencies matter

A real timestamp column (like order_timeorder_time from the previous page) is often irregular — orders don’t arrive exactly every hour. But for analysis you frequently want a fixed-frequency index: every day, every business day, every 15 minutes. Pandas gives you two tools for this:

  • pd.date_rangepd.date_range — generate a clean, evenly spaced DatetimeIndexDatetimeIndex.
  • .shift().shift() — move values (or timestamps) forward/backward along that index.

Think of date_rangedate_range as drawing the ruler, and shiftshift as sliding your data along it.

Generating date ranges with date_rangedate_range

Basic date_range
import pandas as pd
 
# Start + end (daily by default)
idx = pd.date_range("2024-01-01", "2024-01-06")
print(idx)
 
# Start + number of periods
idx2 = pd.date_range(start="2024-01-01", periods=5)
print(idx2)
 
# Business days only (skips weekends)
idx3 = pd.date_range("2024-01-01", periods=5, freq="B")
print(idx3)
Basic date_range
import pandas as pd
 
# Start + end (daily by default)
idx = pd.date_range("2024-01-01", "2024-01-06")
print(idx)
 
# Start + number of periods
idx2 = pd.date_range(start="2024-01-01", periods=5)
print(idx2)
 
# Business days only (skips weekends)
idx3 = pd.date_range("2024-01-01", periods=5, freq="B")
print(idx3)

date_rangedate_range always needs at least two of: startstart, endend, periodsperiods. Give it any two and it works out the rest.

Frequency strings and multiples

Frequencies combine a base alias with an optional multiplier:

AliasMeansAliasMeans
DDCalendar dayMEMEMonth end
BBBusiness dayMSMSMonth start
hhHourQEQEQuarter end
minminMinuteYEYEYear end
ssSecondW-MONW-MONWeekly on Monday
Frequency multiples
import pandas as pd
 
print(pd.date_range("2024-01-01", periods=4, freq="4h"))    # every 4 hours
print(pd.date_range("2024-01-01", periods=4, freq="5min"))  # every 5 minutes
print(pd.date_range("2024-01-01", periods=4, freq="ME"))    # last day of each month
Frequency multiples
import pandas as pd
 
print(pd.date_range("2024-01-01", periods=4, freq="4h"))    # every 4 hours
print(pd.date_range("2024-01-01", periods=4, freq="5min"))  # every 5 minutes
print(pd.date_range("2024-01-01", periods=4, freq="ME"))    # last day of each month

Shifting data: leading and lagging

shiftshift moves the values, keeping the index fixed — this creates NaNNaN at the start or end, which is exactly what you want for lag/lead features:

shift() basics
import pandas as pd
 
ts = pd.Series(
    [100, 105, 102, 108],
    index=pd.date_range("2024-01-31", periods=4, freq="ME"),
)
 
print(ts.shift(1))   # lag by 1 period — value moves down
print(ts.shift(-1))  # lead by 1 period — value moves up
shift() basics
import pandas as pd
 
ts = pd.Series(
    [100, 105, 102, 108],
    index=pd.date_range("2024-01-31", periods=4, freq="ME"),
)
 
print(ts.shift(1))   # lag by 1 period — value moves down
print(ts.shift(-1))  # lead by 1 period — value moves up

A very common use of shiftshift is computing period-over-period percent change:

Percent change via shift
pct_change = ts / ts.shift(1) - 1
print(pct_change)
Percent change via shift
pct_change = ts / ts.shift(1) - 1
print(pct_change)

If you pass a freqfreq, shiftshift moves the timestamps instead of the values, so no data is discarded:

Shifting the index itself
print(ts.shift(2, freq="ME"))  # every date pushed forward by 2 months
Shifting the index itself
print(ts.shift(2, freq="ME"))  # every date pushed forward by 2 months

Rolling dates forward with offsets

Pandas offset objects understand calendar rules. Adding MonthEnd()MonthEnd() to any date “rolls forward” to the next month end — handy for snapping arbitrary dates onto a frequency grid:

Date offsets
import pandas as pd
from pandas.tseries.offsets import MonthEnd, Day
 
now = pd.Timestamp("2024-01-17")
 
print(now + 3 * Day())        # simple: 3 calendar days later
print(now + MonthEnd())       # rolls forward to 2024-01-31
print(now + MonthEnd(2))      # rolls forward 2 month-ends: 2024-02-29
 
offset = MonthEnd()
print(offset.rollforward(now))  # 2024-01-31
print(offset.rollback(now))     # 2023-12-31 (previous month end)
Date offsets
import pandas as pd
from pandas.tseries.offsets import MonthEnd, Day
 
now = pd.Timestamp("2024-01-17")
 
print(now + 3 * Day())        # simple: 3 calendar days later
print(now + MonthEnd())       # rolls forward to 2024-01-31
print(now + MonthEnd(2))      # rolls forward 2 month-ends: 2024-02-29
 
offset = MonthEnd()
print(offset.rollforward(now))  # 2024-01-31
print(offset.rollback(now))     # 2023-12-31 (previous month end)

Common pitfalls

  • Naive shift()shift() (no freqfreq) introduces NaNNaN at the start or end — that’s expected, not a bug.
  • Legacy aliases "M""M", "T""T", "H""H", "A""A" are no longer accepted — use "ME""ME", "min""min", "h""h", "YE""YE".
  • date_rangedate_range needs exactly two of startstart / endend / periodsperiods — passing all three (when inconsistent) or only one raises an error.

Visualize it

diagram From ruler to shifted series mermaid
date_range lays down a fixed-frequency ruler; shift then slides data along that ruler, leaving gaps at the edges.
sketch Shifting a timestamp series p5.js
The amber dots are the original values on the ruler; the blue dots show the same values shifted one step to the right.

🧪 Try It Yourself

Exercise 1 – Build a Fixed-Frequency Range

Exercise 2 – Lag a Series With shift

Exercise 3 – Roll a Date Forward to Month End

Next

Fixed-frequency ranges become tricky once you cross regions — see Time Zone Handling for how to localize and convert timestamps safely.

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did