Concurrency: Threads, Processes & Interpreters

Concurrency, parallelism, and the shape of your workload

Concurrency is about structure: a program has several tasks in progress at the same time, and their steps interleave. Parallelism is about execution: several steps literally run at the same instant on different CPU cores. A single-core machine can run a concurrent program (by switching between tasks) but cannot run it in parallel. Rob Pike's summary: concurrency is dealing with lots of things at once; parallelism is doing lots of things at once.

Which one you need depends on where your program spends its time:

Before parallelizing, estimate the ceiling. Amdahl's law says that if a fraction p of the runtime can be parallelized across n workers, the best possible speedup is 1 / ((1 - p) + p / n). The serial part dominates quickly: with 90% parallel code, even infinite cores give at most 10×.

Real speedups are lower still, because parallel workers pay coordination costs: starting workers, copying data to them, waiting for locks, and combining results. A recurring theme of this module is that each tool sits at a different point on the trade-off between isolation (safety, but expensive data transfer) and sharing (cheap data access, but you must synchronize).

The GIL, explained

CPython manages memory with reference counting: every object has a counter of how many references point at it, and the object is freed when it drops to zero. Incrementing and decrementing those counters happens constantly, on every assignment and function call. If two threads updated the same counter at once without synchronization, counts would be corrupted and objects freed while still in use.

The classic solution is the Global Interpreter Lock (GIL): one mutex per interpreter that a thread must hold to execute Python bytecode or touch Python objects. It makes reference counting and the internals of dict, list, and friends safe without fine-grained locks, and makes single-threaded code fast. The cost: in a standard build, only one thread executes Python bytecode at a time, so threads never speed up pure-Python CPU work.

Threads still help with I/O because the GIL is released at the right moments:

• Blocking system calls: socket reads, file I/O, time.sleep, waiting on a lock or a subprocess. The thread drops the GIL while the OS does the waiting.
• C extensions that opt in: NumPy's large array operations, hashlib on big buffers, zlib/compression.zstd, many image libraries. Their inner loops don't touch Python objects, so they run in parallel with other threads.
• Periodically: a thread running pure Python is asked to drop the GIL every switch interval (5 ms by default, see sys.getswitchinterval()) so other threads get a turn. This is why threads are preemptive: a switch can happen between almost any two bytecode instructions.

That last point is the root of race conditions. The GIL makes individual bytecode instructions atomic, but a line like counter += 1 compiles to several instructions: load, add, store. A switch between the load and the store loses an update.

A thread switch after LOAD_GLOBAL counter but before STORE_GLOBAL counter lets another thread read the same old value. Both store old + 1, and one increment vanishes. In a GIL build this is rare for such a tiny window (and recent CPython versions make it rarer still), which is exactly what makes it dangerous: tests pass, production loses data. In a free-threaded build it happens constantly.

Threads and race conditions

A thread is an independent flow of execution inside one process. All threads share the same memory: the same modules, globals, and objects. That makes communication free and bugs easy. The threading module wraps the operating system's threads (each Python thread is a real OS thread).

The timeline below compares running three I/O waits sequentially with running them in threads. Notice that the threads overlap their waiting, which is where the time goes in I/O-bound work.

A race condition you can watch

Real threads can't run in your browser, but the essence of a race is just interleaving. We can simulate a preemptive scheduler with generators: each yield marks a point where the "OS" may switch to another task. Run it a few times; change the seed; then turn on USE_LOCK.

With the lock off, both withdrawals can be approved against the same balance of 100, which a bank would not enjoy. The lock makes the read-check-write sequence a critical section: only one task may be inside it at a time. Here is the same bug with real threads (the time.sleep(0) forces a switch inside the window to make the race reproducible):

The synchronization toolbox

Remember: threads share memory; any compound operation on shared state needs a critical section. Prefer designs that share less (next section) over designs with more locks.

Producer/consumer with queue.Queue

The safest way to share data between threads is to not share it: pass it. queue.Queue is a thread-safe FIFO with the locking done for you. Producers put() work items; consumers get() them. A maxsize gives back-pressure: when the queue is full, producers block instead of piling up unbounded memory. task_done() and join() let the main thread wait until every item has been processed, and a sentinel (often None) tells consumers to exit.

The API itself works fine without threads, so you can explore it here:

concurrent.futures: executors and futures

Managing threads by hand is verbose. concurrent.futures provides one interface over three kinds of worker pools: ThreadPoolExecutor, ProcessPoolExecutor, and (new in 3.14) InterpreterPoolExecutor. You submit callables and get back futures: placeholder objects for a result that doesn't exist yet.

• executor.submit(fn, *args) returns a Future. future.result(timeout=None) blocks until it's done and returns the value or re-raises the worker's exception.
• executor.map(fn, iterable) returns results in input order (lazily).
• as_completed(futures) yields futures as they finish, which is best for progress bars and for acting on fast results first.
• wait(futures, return_when=FIRST_EXCEPTION) gives you (done, not_done) sets.
• with Executor(...) as ex: calls shutdown(wait=True) on exit; shutdown(cancel_futures=True) drops pending work.

A Future is just a thread-safe box with callbacks; you can drive one by hand to see its lifecycle:

multiprocessing: true parallelism with isolation

A process has its own memory space and its own interpreter (and so its own GIL). Running Python in several processes gives real CPU parallelism on a standard build. The price is isolation: processes don't share objects, so arguments and results must be serialized with pickle, sent through a pipe, and deserialized on the other side. That means:

• Functions and arguments must be picklable. Module-level functions pickle (by reference to their qualified name); lambdas, nested functions, open files, sockets, and locks don't.
• Sending large data is expensive. Parallelizing a task that takes 1 ms but ships 10 MB is a slowdown.
• Workers must be able to import your function, which is why the if __name__ == "__main__": guard is mandatory with the spawn and forkserver start methods. Without it, each child re-runs your script's top level as it imports it.

Start methods

How a child process is created matters for speed, safety, and what state it inherits:

Sharing memory instead of copying it

For big numeric data, pickling every chunk is wasteful. multiprocessing.shared_memory.SharedMemory creates a named block of memory that several processes map; wrap it in a NumPy array and workers read and write the same bytes with no copying. You are back to shared state, so coordinate writes (each worker owns a slice, or use a multiprocessing.Lock). Simpler options: multiprocessing.Value/Array for small typed buffers, and Manager() for proxied dicts and lists (convenient but slow: every access is an IPC round-trip).

Free-threaded Python 3.14: no GIL

PEP 703 introduced an optional CPython build without the GIL in 3.13 (experimental). In 3.14, PEP 779 promoted it to officially supported (phase II): it is a separate build you choose, not the default. In this build, threads run Python bytecode truly in parallel on multiple cores. To make that safe, CPython uses biased reference counting (cheap counts for the owning thread, atomic ones for others), immortal objects (like None and small ints, whose counts never change), per-object locks inside built-in containers, and a thread-safe memory allocator (mimalloc).

The interpreter binary is conventionally called python3.14t (the t stands for "threaded"). You can tell which build you're on, and whether the GIL is actually off right now:

The GIL can come back. If you import a C extension that hasn't declared itself free-threading compatible, the interpreter re-enables the GIL (and warns) to stay safe. You can force the choice with the environment variable PYTHON_GIL=0 or PYTHON_GIL=1, or with -X gil=0. Most of the scientific stack (NumPy, SciPy, and others) publishes free-threaded builds, but check each dependency.

Thread safety without a GIL

Free threading doesn't change Python's semantics, but it removes the accidental protection many programs relied on. What still holds and what doesn't:

Performance expectations

• Single-threaded code is somewhat slower on the free-threaded build (the 3.14 docs put the overhead at roughly 5–10% for typical code, down from much higher in 3.13), because of atomic reference counts and fine-grained locking. Some specializations of the adaptive interpreter are also more limited.
• CPU-bound multi-threaded Python code scales with cores, which is the whole point: n threads on n cores can approach n× for embarrassingly parallel work.
• Contention kills scaling: threads hammering one shared dict, list, or lock serialize on its per-object lock. Partition data so each thread works on its own objects and combine at the end.
• Memory use is somewhat higher, and extensions that re-enable the GIL silently remove all the gains. Measure on your workload with the benchmark in the mini-project.

Subinterpreters: concurrent.interpreters

A third model sits between threads and processes. A subinterpreter is a separate Python interpreter inside the same process: its own modules, its own sys, its own objects, and, since 3.12 (PEP 684), its own GIL. Several subinterpreters running in different threads execute Python truly in parallel, even on a standard GIL build. Python 3.14 exposes this through the new concurrent.interpreters module (PEP 734) and concurrent.futures.InterpreterPoolExecutor.

Choosing the right tool

Practice

def speedup(p, n):
    if not 0 <= p <= 1 or n < 1:
        raise ValueError("need 0 <= p <= 1 and n >= 1")
    return 1 / ((1 - p) + p / n)

def workers_needed(p, target):
    if p < 1 and target >= 1 / (1 - p) - 1e-12:
        return None
    n = 1
    while speedup(p, n) < target - 1e-12:
        n += 1
    return n

def run_round_robin(tasks):
    live = list(tasks)
    while live:
        for task in list(live):
            try:
                next(task)
            except StopIteration:
                live.remove(task)

def simulate(n_tasks, increments, atomic):
    state = {"count": 0}

    def worker():
        for _ in range(increments):
            if atomic:
                state["count"] += 1
                yield
            else:
                value = state["count"]
                yield                      # switch point inside read-modify-write
                state["count"] = value + 1

    run_round_robin([worker() for _ in range(n_tasks)])
    return state["count"]

def chunk_ranges(n, workers):
    if workers < 1 or n < 0:
        raise ValueError("need workers >= 1 and n >= 0")
    k = min(workers, n)
    if k == 0:
        return []
    size, extra = divmod(n, k)
    chunks, start = [], 0
    for i in range(k):
        stop = start + size + (1 if i < extra else 0)
        chunks.append((start, stop))
        start = stop
    return chunks

from concurrent.futures import Future, CancelledError

def collect(futures):
    results, errors = [], []
    for i, f in enumerate(futures):
        if f.cancelled():
            errors.append((i, CancelledError()))
            continue
        exc = f.exception()
        if exc is None:
            results.append(f.result())
        else:
            errors.append((i, exc))
    return results, errors

Mini-project: a parallelism benchmark

pixi init bench-parallel
cd bench-parallel
pixi add python=3.14 numpy
pixi add --feature ft python-freethreading
pixi workspace environment add ft --feature ft
pixi run python bench.py --workers 1 2 4 8
pixi run -e ft python bench.py --workers 1 2 4 8

Quiz

Recap

• Concurrency is structure; parallelism is simultaneous execution. Identify I/O-bound vs CPU-bound work first, and bound your expectations with Amdahl's law.
• The GIL protects interpreter internals, is released during blocking I/O and by many C extensions, and never made your code thread-safe.
• Threads share memory: protect compound operations with Lock/RLock/Condition, or pass data through queue.Queue.
• concurrent.futures gives one API over thread, process, and interpreter pools.
• Processes are isolated and parallel but pay for pickling; 3.14 defaults to forkserver on Linux; use shared memory for big arrays.
• Free-threaded 3.14 (python3.14t) is officially supported: true thread parallelism, ~5–10% single-thread cost, and your own thread-safety discipline.
• concurrent.interpreters and InterpreterPoolExecutor give isolated parallelism inside one process.

Going further

• Python support for free threading (HOWTO) and py-free-threading.github.io
• concurrent.interpreters and PEP 734
• multiprocessing: contexts and start methods
• threading, queue, concurrent.futures
• Next: M38 Async Programming with asyncio