Files and Structured Data

Why it matters

Most real programs eventually read a configuration file, write a report, or process data someone else produced. Getting comfortable with paths, safe file handling, and the two most common structured formats — JSON and CSV — is what turns “a script that only works on my machine with hardcoded values” into something that actually runs reliably elsewhere.

Mental model

Three ideas do almost all the work in this lesson:

• pathlib.Path represents a filesystem path as an object, so joining paths (directory / "config.json") works correctly on Windows and POSIX systems alike, instead of hand-building strings with the wrong separator.
• Context managers (with) guarantee cleanup — closing a file — runs whether the code inside succeeded or raised an exception.
• JSON and CSV are text formats for structured data: JSON for nested objects/arrays/values, CSV for tabular rows. Both are naive about types — everything read from CSV is a string until you convert it yourself, and JSON has no native representation for dates or arbitrary Python objects.

Worked example: a config-driven expense summary

import csv
import json
import tempfile
from pathlib import Path


def write_config(directory: Path) -> Path:
    config = {"currency": "USD", "categories": ["food", "transport", "other"]}
    config_path = directory / "config.json"
    config_path.write_text(json.dumps(config, indent=2), encoding="utf-8")
    return config_path


def write_expenses(directory: Path) -> Path:
    rows = [
        {"category": "food", "amount": "12.50"},
        {"category": "transport", "amount": "8.00"},
        {"category": "food", "amount": "5.25"},
    ]
    expenses_path = directory / "expenses.csv"
    with expenses_path.open("w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=["category", "amount"])
        writer.writeheader()
        writer.writerows(rows)
    return expenses_path


def summarize(config_path: Path, expenses_path: Path) -> dict[str, float]:
    config = json.loads(config_path.read_text(encoding="utf-8"))
    totals = {category: 0.0 for category in config["categories"]}

    with expenses_path.open(encoding="utf-8") as f:
        reader = csv.DictReader(f)
        for row in reader:
            totals[row["category"]] += float(row["amount"])

    return totals


def main() -> None:
    with tempfile.TemporaryDirectory() as tmp:
        directory = Path(tmp)
        config_path = write_config(directory)
        expenses_path = write_expenses(directory)

        totals = summarize(config_path, expenses_path)
        for category, total in totals.items():
            print(f"{category}: {total:.2f}")


if __name__ == "__main__":
    main()

food: 17.75
transport: 8.00
other: 0.00

write_config uses Path.write_text, which opens, writes, and closes in one call — no explicit with needed for that simple case. write_expenses uses with expenses_path.open(...) as f: directly, because csv.DictWriter needs a real, still-open file object to write rows into as it goes. summarize reads the config to know which categories to expect (other ends up 0.00 because no expense row used it — the config, not the CSV, decides the full set of categories), then accumulates CSV amounts — converting each row["amount"] from str to float explicitly, since every value read from a CSV row is a string, regardless of what it looks like.

Common mistakes

• Building paths with string concatenation (folder + "/" + filename) instead of Path — this silently breaks on Windows, where the native separator is \, not /.
• Forgetting that every value read from a CSV row is a str — row["amount"] + 1 concatenates strings or raises TypeError, depending on the type of 1; it never does arithmetic until the value is explicitly converted.
• Opening a file without with and relying on it eventually being closed, especially across a code path that might raise an exception before an explicit close() call runs.
• Committing a config file that happens to contain a real secret because it was convenient during development — treat any file checked into source control as effectively public.

Check your understanding

Practice: warm-up

Predict, then check: does Path("data") / "reports" / "q1.csv" work identically on Windows and on macOS/Linux, even though those systems use different path separators?

from pathlib import Path

p = Path("data") / "reports" / "q1.csv"
print(p)

# On Windows:        data\reports\q1.csv
# On macOS/Linux:     data/reports/q1.csv

Practice: apply it

Extend summarize (or write a similar function) so it also reports the count of expenses per category, not just the total — returning dict[str, tuple[float, int]] instead of dict[str, float]. Predict the count for "food" given this lesson’s sample data before checking.

Summary

• pathlib.Path builds correct, cross-platform paths with /; prefer it over manual string concatenation.
• A context manager (with) guarantees cleanup runs whether its block succeeds or raises — the standard way to handle an open file safely.
• Every value read from a CSV row is a str; JSON preserves basic types (numbers, strings, booleans, null, nested objects/arrays) but has no native representation for dates or custom Python objects.
• Configuration can live in a file; secrets belong in environment variables or a secrets manager, never in a file that gets committed to source control.