Text and String Methods (str accessor and regex)

Why not just use Python string methods?

Python's built-in string methods (.lower(), .split(), .replace()) work great on a single string. But a DataFrame column can hold thousands of strings — and, almost always, a few missing values mixed in. Calling .lower() directly on every value with .map() works fine until it hits a NaN, and then it crashes.

Pandas solves this with the str accessor: series.str.lower() runs a vectorized, element-wise version of the string method across the whole column, and automatically skips (propagates) missing values instead of erroring out.

Example dataset

import pandas as pd
 
emails = pd.Series(
    ["Dave@Google.com", "steve@gmail.com ", None, "  rob@gmail.com"],
    index=["Dave", "Steve", "Wes", "Rob"],
)
 
print(emails)
print(emails.isna())

Everyday cleanup: lower, strip, contains, replace

print(emails.str.lower())
print(emails.str.strip().str.lower())

str.contains checks each value for a substring (or pattern) and returns a boolean Series — pass na=False so missing values count as "no match" instead of showing up as NaN in a mask:

has_gmail = emails.str.contains("gmail", case=False, na=False)
print(has_gmail)
print(emails[has_gmail])

str.replace swaps out a substring everywhere it appears — pass regex=False when your search text isn't a pattern, so characters like . aren't treated as "any character":

print(emails.str.replace("gmail", "GMAIL", regex=False))

Splitting and pulling pieces out

str.split breaks each string into a list at a delimiter; indexing into the result with .str[i] or .str.get(i) pulls out one piece of each list:

parts = emails.str.strip().str.split("@")
print(parts)
print(parts.str.get(0))   # username, before the @
print(parts.str.get(1))   # domain, after the @

Slicing works too, the same way you'd slice a plain Python string:

print(emails.str.strip().str[:4])

Regex with .str.extract and .str.findall

For real pattern matching — not just a fixed substring — pass a regular expression. str.extract pulls out groups (the parts in parentheses) into their own columns; str.findall returns every match as a list:

import re
 
pattern = r"([A-Za-z0-9._%+-]+)@([A-Za-z0-9.-]+)\.([A-Za-z]{2,4})"
 
clean_emails = emails.str.strip()
print(clean_emails.str.extract(pattern))

The result is a small DataFrame with one column per capture group — column 0 is the username, 1 is the domain, 2 is the suffix. Rows that don't match (or were NaN to begin with) come back as NaN in every column, not an error.

print(clean_emails.str.findall(pattern, flags=re.IGNORECASE))

Common pitfalls

• Forgetting regex=False on str.replace when your replacement text contains regex special characters like . or ( — it will silently match more than you intended.
• Boolean masks from str.contains can contain NaN (for missing input) unless you pass na=False — filtering with a mask that still has NaN in it raises an error.
• str.extract needs parentheses (...) around the parts you want back; a pattern with no groups extracts nothing useful.

Visualize it

  flowchart LR
  A["Series with NaN mixed in"] --> B[".str accessor"]
  B --> C["Real values -> string/regex logic"]
  B --> D["NaN -> stays NaN"]
  C --> E["Cleaned Series"]
  D --> E

🧪 Try It Yourself

Exercise 1 – Lowercase an Entire Column

Press Run to see what your code prints.

Exercise 2 – Check Membership While Ignoring NaN

Press Run to see what your code prints.

Exercise 3 – Extract a Regex Group

Press Run to see what your code prints.

Check your understanding

Next

With text under control, move on to Reading and Writing Data (CSV, Excel, JSON) to load real datasets that will need exactly this kind of cleanup.