Processing CSV Data with the csv Module

Reading CSV

import csv
 
with open("input.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row)

Writing CSV

import csv
 
rows = [
    {"name": "Alice", "score": 95},
    {"name": "Bob", "score": 88},
]
 
with open("out.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "score"])
    writer.writeheader()
    writer.writerows(rows)

Common cleanup

• strip whitespace
• handle missing values
• normalize casing

Why use the csv module

A CSV file looks like text you could split on commas, but that fails quickly. A value can contain a comma ("Smith, Ana"), a quote, or even a line break, and the format handles it with quoting. The csv module implements those rules so you do not have to. It is also in the standard library, so it works everywhere with no install.

reader and DictReader

csv.reader gives each row as a list, so you refer to columns by position. csv.DictReader uses the first row as header names and gives each row as a dictionary. Prefer DictReader: row["score"] still works if someone inserts a new column, while row[3] silently reads the wrong one.

Two arguments matter every time. newline="" lets the module handle line endings itself, and without it you can get blank lines on Windows. encoding="utf-8" fixes the text encoding. For files exported from Excel, try encoding="utf-8-sig", which removes the invisible byte-order mark that otherwise ends up glued to the first header name.

Everything you read is a string. row["score"] is "95", not 95, so convert before you calculate.

A cleaning script

import csv
from pathlib import Path
 
 
def clean_rows(src: Path, dest: Path) -> int:
    kept = 0
    with src.open(newline="", encoding="utf-8-sig") as fin, \
         dest.open("w", newline="", encoding="utf-8") as fout:
        reader = csv.DictReader(fin)
        writer = csv.DictWriter(fout, fieldnames=["name", "score"])
        writer.writeheader()
 
        for row in reader:
            name = (row.get("name") or "").strip().title()
            raw = (row.get("score") or "").strip()
            if not name or not raw:
                continue                      # skip incomplete rows
            try:
                score = int(raw)
            except ValueError:
                print("bad score:", row)
                continue
            writer.writerow({"name": name, "score": score})
            kept += 1
    return kept
 
 
print(clean_rows(Path("input.csv"), Path("clean.csv")), "rows kept")

It strips whitespace, normalises the case of names, drops rows with missing values, reports rows with a non-numeric score, and writes to a new file. The input is never modified. row.get("name") or "" handles both a missing column and an empty cell.

Other delimiters

Some files use semicolons or tabs. Pass delimiter=";" or delimiter="\t" to the reader and writer. csv.Sniffer can guess, but a wrong guess on a small sample is possible, so set it explicitly when you know.

Common mistakes

• Forgetting newline="" and getting blank lines between rows on Windows.
• Treating numbers as numbers. Values are strings until you convert them.
• Writing to the same file you are reading. Write to a new file, then replace the original if needed.
• Splitting lines on commas by hand.
• Assuming every row has every column. DictReader fills missing values with None.

Adapting it

Change the columns and the cleaning rules to fit your data. For big analysis, hand the file to pandas. For a report, pass the cleaned rows to openpyxl, as in the reports lesson.

🧪 Try It Yourself

Exercise 1 – List Files with os.listdir

Press Run to see what your code prints.

Exercise 2 – Join Paths with os.path.join

Press Run to see what your code prints.

Exercise 3 – Write and Read a File

Press Run to see what your code prints.