Code Quality & Tooling

Why automate quality?

Humans are bad at consistency and good at judgement. Code review time is expensive, so it shouldn't be spent on "missing blank line" or "unused import". A quality harness is a set of tools that run automatically and refuse to let a change through until it meets the bar. This leaves reviewers free to ask the questions that need a person: is this the right design, is it correct, is it secure?

The harness runs at three points, from fastest to slowest:

flowchart LR
  E[Editor
ruff + mypy on save] --> P[pre-commit
on git commit]
  P --> L[pixi run ci
full local gate]
  L --> G[GitHub Actions
matrix CI on every PR]
  G --> M[Merge to main]
    

This repository's gate is one Pixi task, ordered so the quickest failures surface first:

fmt = "ruff format ."
lint = "ruff check ."
check = "mypy src/"
cov = "pytest tests/ --cov=src --cov-report=term-missing --cov-fail-under=90"
build = "python -m build --no-isolation"
pre-commit-all = "pre-commit run --all-files"
ci = { depends-on = ["pre-commit-all", "build", "check", "lint", "cov"] }

Remember: automate everything a machine can check, run the fast checks first, and save human review for design and correctness.

ruff: linting and formatting

A linter reads your code without running it and reports likely bugs and style problems. A formatter rewrites layout (spacing, line breaks, quotes) into one canonical style, which ends formatting debates for good. ruff does both. It's written in Rust and is fast enough to run on every keystroke, and it reimplements hundreds of rules from older tools (flake8, isort, pyupgrade, pydocstyle, Black).

Rules are grouped by a letter prefix. This repo selects seven groups:

[tool.ruff]
line-length = 120
src = ["src", "tests"]
extend-exclude = ["work"]          # learners' scratch copies are not linted

[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "RUF", "D"]
ignore = ["E501"]                  # the formatter owns line length

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["D100", "D103", "D104"]   # tests don't need module/function docstrings
"projects/**/tests/**" = ["D100", "D103", "D104"]

[tool.ruff.lint.pydocstyle]
convention = "google"

[tool.ruff.lint.isort]
known-first-party = ["practicum"]

Here's real ruff output on a small file with several common mistakes:

"""Inventory helpers."""
import os, sys
from typing import List, Optional


def add_item(item, bucket=[]):
    bucket.append(item)
    return bucket


def find(items: List[str], name: str) -> Optional[str]:
    """Find an item."""
    for i in range(len(items)):
        if items[i] == name:
            return items[i]
    try:
        return None
    except:
        pass

$ pixi run ruff check inventory.py --output-format concise
inventory.py:2:1: E401 [*] Multiple imports on one line
inventory.py:2:1: I001 [*] Import block is un-sorted or un-formatted
inventory.py:2:8: F401 [*] `os` imported but unused
inventory.py:2:12: F401 [*] `sys` imported but unused
inventory.py:3:1: UP035 `typing.List` is deprecated, use `list` instead
inventory.py:6:5: D103 Missing docstring in public function
inventory.py:6:27: B006 Do not use mutable data structures for argument defaults
inventory.py:11:17: UP006 [*] Use `list` instead of `List` for type annotation
inventory.py:11:42: UP045 [*] Use `X | None` for type annotations
inventory.py:18:5: E722 Do not use bare `except`
Found 10 errors.
[*] 6 fixable with `--fix` (1 unsafe fix available with `--fix --unsafe-fixes`).

$ pixi run ruff check --fix .     # apply the safe fixes ([*])
$ pixi run ruff format .          # reformat everything
$ pixi run ruff rule B006         # read the full explanation of a rule

B006 isn't about style. It's a real bug, and you can see it happen:

A linter is "just" a program that walks the abstract syntax tree (AST). This toy one, written with the standard ast module, detects B006 and F401. Try adding more violations:

Remember: ruff check finds problems, --fix repairs the safe ones, and ruff format ends style debates. Choose rule groups with select and scope any exceptions with per-file-ignores.

mypy in strict mode

Python ignores type hints at runtime (M16). mypy is a static type checker: it reads the hints and proves, before anything runs, that values flow consistently. Plain mypy is lenient, and it skips functions without annotations entirely. Strict mode turns on the flags that make the guarantee mean something:

• disallow_untyped_defs: every function must be annotated.
• disallow_any_generics: no bare list; say list[str].
• warn_return_any: don't silently return Any from a typed function.
• no_implicit_reexport, strict_equality, warn_unused_ignores, and more.

[tool.mypy]
python_version = "3.12"   # the oldest supported Python, so newer-only APIs are flagged
strict = true
warn_return_any = true
warn_unused_configs = true
files = ["src"]

$ pixi run mypy --strict inventory.py
inventory.py:6: error: Function is missing a type annotation  [no-untyped-def]
inventory.py:11: error: Missing return statement  [return]
inventory.py:24: error: Incompatible return value type (got "float", expected "int")  [return-value]
Found 3 errors in 1 file (checked 1 source file)

The third error is a bug that no test may have caught yet. total() claims to return int but returns a float. The interpreter doesn't care at all, which is exactly why you want a checker:

Remember: hints are only as trustworthy as the checker that enforces them. Strict mypy in CI turns type hints into guarantees.

pre-commit hooks and the restage trap

Git can run scripts at certain moments. A pre-commit hook runs before a commit is recorded, and if it fails, the commit is refused. The pre-commit framework manages these hooks: it installs each tool at a pinned version in its own environment and runs it on the files you're committing.

repos:
  - repo: https://github.com/compilerla/conventional-pre-commit
    rev: v4.4.0
    hooks:
      - id: conventional-pre-commit
        stages: [commit-msg]          # checks the message, not the code
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.15.17
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v2.1.0
    hooks:
      - id: mypy
        files: ^src/
        additional_dependencies: [structlog]

pixi run bootstrap                 # pre-commit install (+ the commit-msg hook type)
pixi run pre-commit-all            # pre-commit run --all-files
pixi run pre-commit autoupdate     # bump every rev to the latest release (review the diff!)

The auto-fix restage trap. Git commits what's in the staging area (the index), not what's in your working files. When a hook fixes a file (ruff --fix removing an import, ruff-format re-wrapping a line), it writes the fix to your working copy and then fails with "files were modified by this hook". The staged version is still the unfixed one. If you just run git commit again without restaging, you're committing the same broken content and you'll get the same failure.

Remember: pre-commit runs pinned tools on staged files. When a hook auto-fixes something, git add the fixed files and commit again.

Structured logging with structlog

In beginner code, print() is how you see what's happening. In production code, never print. Here's why:

• print writes to stdout, which may be your program's output (a CLI that emits JSON, a pipe). Diagnostics mixed into it corrupt that output.
• It has no levels, so you can't turn debug chatter off in production or turn it on while investigating.
• It produces prose like "User 42 bought 3 items", which a log aggregator (Datadog, Loki, CloudWatch, ELK) can only search with fragile regexes.
• It carries no context: no timestamp, request id, or module.

The stdlib logging module solves levels and destinations, but its records are still format strings, its configuration is global and fiddly, and attaching context needs adapters and filters. This course uses structlog. Every log call is an event plus key-value pairs, and it flows through a chain of processors (plain functions) that enrich it and finally render it as JSON or key=value text.

A processor has the signature processor(logger, method_name, event_dict) -> event_dict. The last one returns a string. Because the whole design is just functions passing dicts along, you can rebuild it in a few lines and watch it work. structlog itself isn't available in the browser, so this cell is a faithful miniature:

The real thing, as configured in this repository's src/practicum/log.py, renders key=value for humans and JSON for machines:

def configure_logging(*, json: bool = False) -> None:
    """Configure structlog for key=value console output or JSON output."""
    renderer: structlog.typing.Processor = (
        structlog.processors.JSONRenderer() if json else structlog.dev.ConsoleRenderer(colors=sys.stderr.isatty())
    )
    structlog.configure(
        processors=[
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(fmt="iso"),
            renderer,
        ],
        logger_factory=structlog.PrintLoggerFactory(file=sys.stderr),   # logs go to stderr, never stdout
        cache_logger_on_first_use=False,
    )

A fuller production setup adds context variables (so a request id set once appears on every line in that request), level filtering, and exceptions rendered as structured data:

{"user_id": 42, "order_id": 1001, "total": 59.9, "event": "order_placed", "request_id": "r-81", "level": "info", "timestamp": "2026-10-15T01:06:53.648248Z"}
{"user_id": 42, "event": "payment_failed", "request_id": "r-81", "level": "error", "timestamp": "...", "exception": [{"exc_type": "ZeroDivisionError", "exc_value": "division by zero", "frames": [...]}]}

• Event names are identifiers, not sentences: order_placed, not "Order was placed!". You'll search and count by them.
• Data goes in keys, not in the event string: log.info("order_placed", order_id=1001).
• bind() returns a new logger with extra context. bind_contextvars sets context for the current thread or async task, which suits request ids in web servers.
• Configure once, at the program's entry point (main()). Libraries only call structlog.get_logger() and never configure.
• Dev vs prod: colourful ConsoleRenderer locally, JSONRenderer in production, chosen by a flag or an environment variable.

Remember: production code logs events with key-value data through a processor chain, configured once in main(). Render JSON in production and readable text in development, and never use print().

Conventional Commits and git-cliff changelogs

Conventional Commits is a small grammar for commit messages that both humans and tools can read:

type(scope)!: short description in the imperative

optional body explaining what and why

BREAKING CHANGE: optional footer describing the incompatibility
Refs: #123

feat(cli): add --json output to textstats
fix(grade): report a missing project id instead of crashing
docs: explain the restage trap in CONTRIBUTING
refactor(log)!: rename configure() to configure_logging()

This repo enforces the format with the conventional-pre-commit hook at the commit-msg stage, and generates CHANGELOG.md with git-cliff (pixi run changelog). Its configuration lives in pyproject.toml:

[tool.git-cliff.git]
conventional_commits = true
filter_unconventional = true
commit_parsers = [
    { message = "^feat", group = "Features" },
    { message = "^fix", group = "Bug Fixes" },
    { message = "^doc", group = "Documentation" },
    { message = "^perf", group = "Performance" },
    { message = "^refactor", group = "Refactoring" },
    { message = "^chore", skip = true },
]

The parsers are regular expressions tried in order: the first match decides the group. Here's the same logic in Python:

Remember: use the form type(scope): imperative description. With that grammar, tools can write the changelog for you and work out the next version number from the history.

Branches and pull-request hygiene

main should always be releasable. Work happens on short-lived branches whose name says what kind of change it is, and each change reaches main through a pull request (PR) that CI and a reviewer approve:

git switch main && git pull --ff-only
git switch -c feature/add-json-output
# ... edit, pixi run test, pixi run fmt, pixi run lint && pixi run check ...
pixi run ci                                  # must exit 0
git add -p                                   # stage deliberately, hunk by hunk
git commit -m "feat(cli): add --json output to textstats"
git push -u origin feature/add-json-output
gh pr create --fill
gh pr checks --watch                         # wait for every check to go green
gh pr merge --squash --delete-branch
git switch main && git pull --ff-only && git fetch --prune

Remember: keep main releasable, use one branch and one PR per concern, and merge only on green CI.

GitHub Actions CI, act, and coverage gates

Continuous integration (CI) runs the harness on a clean machine for every push and PR, so "it passed on my laptop" is no longer the evidence. GitHub Actions reads YAML workflows from .github/workflows/. The official prefix-dev/setup-pixi action installs Pixi, can cache environments, and installs from your lockfile.

name: CI
on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

jobs:
  unit:
    name: Unit tests (${{ matrix.environment }})
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false                 # let every version report, even if one fails
      matrix:
        environment: [py312, py313, py314]
    steps:
      - uses: actions/checkout@v4
      - uses: prefix-dev/setup-pixi@v0.8.1
        with:
          environments: ${{ matrix.environment }}
          locked: true                 # fail if pixi.lock is stale
          cache: true
      - run: pixi run -e ${{ matrix.environment }} test

  full:
    name: Full CI gate
    runs-on: ubuntu-latest
    needs: unit
    steps:
      - uses: actions/checkout@v4
      - uses: prefix-dev/setup-pixi@v0.8.1
        with:
          locked: true
          cache: true
      - run: pixi run ci

Running CI locally with act. act (installed here through the dev feature) reads the same workflow files and runs the jobs in Docker containers, so you can debug a workflow without push-and-pray:

pixi run act                        # act --container-architecture linux/amd64 (needed on Apple Silicon)
pixi run act -- -j unit             # one job
pixi run act -- pull_request -l     # list the jobs a pull_request event would run

Coverage gates. pytest-cov measures which lines your tests executed. --cov-fail-under=90 makes the run fail below 90%, so coverage can't slowly erode. Coverage tells you what's untested. It can't tell you what's well tested, because a test that runs a line without asserting anything still counts.

$ pixi run cov
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
src/practicum/cli.py         70      4    94%   88-91
src/practicum/site.py       150      9    94%   140-148
-------------------------------------------------------
TOTAL                       610     38    94%
Required test coverage of 90% reached. Total coverage: 93.77%

Remember: CI is the harness run on a clean machine, installed from the locked environment. Make each matrix entry a Pixi environment, debug workflows locally with act, and let the coverage gate stop gradual decline.

Code review checklists

Once the machines have checked formatting, lint, types, and coverage, a reviewer's attention goes to what only a person can judge. A checklist keeps reviews consistent:

• Correctness: does it do what the PR says? What about edge cases (empty input, None, huge input, unicode, time zones)?
• Tests: do they assert behaviour, not implementation details? Would they fail if the code were wrong?
• Errors: are exceptions specific, never bare except, never swallowed silently?
• Security: is user input validated? Any SQL or shell built from strings? Any secrets in code or logs?
• Design: is it the simplest thing that works? Is there duplication of an existing helper?
• Naming and docs: would a newcomer understand it? Are docstrings present on public APIs?
• Logging: structured events with useful keys, no print()?
• Dependencies: is a new package justified, on conda-forge, and added via Pixi?

Remember: machines check style, so people review correctness, tests, security, and design.

Practice

import re

PATTERN = re.compile(r"^([a-z]+)(?:\(([^)]+)\))?(!)?: (.+)$")

def parse_commit(message: str) -> dict | None:
    lines = message.splitlines()
    if not lines:
        return None
    match = PATTERN.match(lines[0])
    if match is None:
        return None
    kind, scope, bang, description = match.groups()
    footer_break = any(line.startswith("BREAKING CHANGE:") for line in lines[1:])
    return {"type": kind, "scope": scope, "breaking": bool(bang) or footer_break, "description": description}

import re

PATTERN = re.compile(r"^([a-z]+)(?:\(([^)]+)\))?(!)?: (.+)$")

def parse_commit(message):
    lines = message.splitlines()
    match = PATTERN.match(lines[0]) if lines else None
    if match is None:
        return None
    kind, scope, bang, desc = match.groups()
    brk = bool(bang) or any(l.startswith("BREAKING CHANGE:") for l in lines[1:])
    return {"type": kind, "scope": scope, "breaking": brk, "description": desc}

def next_version(current: str, messages: list[str]) -> str:
    major, minor, patch = (int(p) for p in current.split("."))
    commits = [c for c in map(parse_commit, messages) if c is not None]
    if any(c["breaking"] for c in commits):
        return f"{major + 1}.0.0"
    if any(c["type"] == "feat" for c in commits):
        return f"{major}.{minor + 1}.0"
    if any(c["type"] in {"fix", "perf"} for c in commits):
        return f"{major}.{minor}.{patch + 1}"
    return current

import re

PATTERN = re.compile(r"^([a-z]+)(?:\(([^)]+)\))?(!)?: (.+)$")
GROUPS = {"feat": "Features", "fix": "Bug Fixes", "docs": "Documentation",
          "perf": "Performance", "refactor": "Refactoring"}

def render_changelog(version: str, messages: list[str]) -> str:
    collected: dict[str, list[str]] = {group: [] for group in GROUPS.values()}
    for message in messages:
        first = message.splitlines()[0] if message else ""
        match = PATTERN.match(first)
        if match is None or match.group(1) not in GROUPS:
            continue
        desc = match.group(4)
        collected[GROUPS[match.group(1)]].append(desc[0].upper() + desc[1:])
    lines = [f"## {version}"]
    for group, items in collected.items():
        if items:
            lines += ["", f"### {group}"] + [f"- {item}" for item in items]
    return "\n".join(lines)

def render_kv(logger, method_name, event_dict):
    keys = ["event"] if "event" in event_dict else []
    keys += sorted(k for k in event_dict if k != "event")
    return " ".join(f"{k}={event_dict[k]!r}" for k in keys)

def import_rows(rows):
    ok = 0
    for i, row in enumerate(rows):
        if "id" not in row:
            print(f"Skipping row {i}: no id")
            continue
        ok += 1
    print(f"Imported {ok} of {len(rows)} rows")
    return ok

def import_rows(rows, log):
    ok = 0
    for i, row in enumerate(rows):
        if "id" not in row:
            log.warning("row_skipped", index=i, reason="missing_id")
            continue
        ok += 1
    log.info("import_finished", imported=ok, total=len(rows))
    return ok

Mini-project: add a full quality harness

Quiz

Recap

• Automate every mechanical check, order it fast-fail first, and leave human review for judgement.
• ruff lints (E F I UP B RUF D) and formats. --fix applies safe fixes and per-file-ignores scopes exceptions.
• Strict mypy turns hints into guarantees. In this harness it runs both in pre-commit and over the full tree.
• pre-commit runs pinned hooks on staged files. After an auto-fix, restage and commit again, and never use --no-verify.
• structlog: events plus key-value pairs through processors, rendered as JSON in production and console text in development. Configure it once in main() and log to stderr, never with print.
• Conventional Commits drive changelogs (git-cliff) and version bumps. Use feature/bugfix/hotfix branches and small PRs.
• CI with setup-pixi installs from the lockfile with each matrix entry as a Pixi environment. Use act to debug locally, and let the coverage gate stop decline.

Going further

• Ruff rules reference
• mypy command line and strict flags
• pre-commit documentation
• structlog documentation
• Conventional Commits 1.0.0 and git-cliff
• prefix-dev/setup-pixi and act user guide
• The ast module