Searching Files by Content/Extension

  flowchart TD
  A["os.walk / rglob"] --> B["prune directories you never want"]
  B --> C{"does the name match?"}
  C -->|no| D["skip -- no file was opened"]
  C -->|yes| E{"searching content too?"}
  E -->|no| F["report it"]
  E -->|yes| G["open it"]
  G --> H{"is it text?"}
  H -->|no| I["UnicodeDecodeError -- catch it, or open in binary"]
  H -->|yes| J["scan line by line, not read()"]
  J --> K["report the path and line number"]

Find files by extension

from pathlib import Path
 
root = Path(".")
for p in root.rglob("*.md"):
    print(p)

Search file contents

This example searches for a word in .py files.

from pathlib import Path
 
 
def search_word(root: Path, word: str):
    for p in root.rglob("*.py"):
        try:
            text = p.read_text(encoding="utf-8")
        except UnicodeDecodeError:
            continue
 
        if word in text:
            print("found in", p)
 
 
search_word(Path("."), "asyncio")

How the search works

Path.rglob("*.py") walks the folder and every subfolder and yields the paths that match the pattern. Filtering by name is cheap because the operating system already gave you the names. Reading content is the expensive part, so the sensible order is to filter by extension first and open only the files that pass.

In search_content.py, read_text decodes the file as UTF-8. Binary files (images, executables) and files in another encoding raise UnicodeDecodeError. The try and except skips them, which is what you want for a search.

Reporting line numbers

Knowing that a word appears in a file is rarely enough. This version reads line by line, so it never loads a huge file into memory, and it prints where each match is. It also skips folders you never want to search.

from pathlib import Path
 
SKIP_DIRS = {".git", "node_modules", "__pycache__", ".venv"}
 
 
def search_lines(root: Path, word: str, pattern: str = "*.py", ignore_case: bool = True):
    needle = word.lower() if ignore_case else word
    for path in root.rglob(pattern):
        if SKIP_DIRS.intersection(path.parts):
            continue
        try:
            with path.open(encoding="utf-8") as f:
                for number, line in enumerate(f, start=1):
                    hay = line.lower() if ignore_case else line
                    if needle in hay:
                        print(f"{path}:{number}: {line.strip()}")
        except (UnicodeDecodeError, PermissionError):
            continue
 
 
search_lines(Path("."), "todo", "*.py")

path.parts is a tuple of the folder names in the path, so intersection tells you whether any skipped folder is in it. enumerate(f, start=1) gives 1-based line numbers, which is what editors show.

Common mistakes

• Reading whole files with read_text on big trees. One 2 GB log file will use 2 GB of memory. Iterate the file object instead.
• Searching .git or node_modules. These hold thousands of files you do not care about and make the search slow. Skip them.
• Forgetting case. "TODO" and "todo" are different strings. Lower-case both sides for a case-insensitive search.
• Treating a pattern as a regex. rglob("*.py") uses glob syntax, not regular expressions. For content matching with patterns, use the re module on each line.

Adapting it

Change the pattern to search other file types. Replace needle in hay with re.search(regex, line) to match patterns such as email addresses. Write the results to a CSV file instead of printing them.

Tips

• skip binary files
• limit walking large folders
• consider ripgrep for huge repos

🧪 Try It Yourself

Exercise 1 – List Files with os.listdir

Press Run to see what your code prints.

Exercise 2 – Join Paths with os.path.join

Press Run to see what your code prints.

Exercise 3 – Write and Read a File

Press Run to see what your code prints.