Choosing between threads, processes and asyncio for a Python workload

Goal

Pick the concurrency model that matches the workload's bottleneck before the design hardens around a wrong one.

Prerequisites

A profile or measurement that says where the time goes on the hot path: waiting on I/O (sockets, disks, subprocesses), executing Python bytecode (parsing, pure-Python number crunching), or running native code that releases the GIL (compression, hashing, many array operations). See "Profile before optimising".

Steps

1. Count the concurrent waits. Thousands of connections, or libraries that are async-native, point to asyncio. A few dozen blocking calls through synchronous libraries point to a ThreadPoolExecutor; the documentation states its default worker count is min(32, cpu_count + 4) (based on os.process_cpu_count() since 3.13), chosen to preserve at least five workers for I/O-bound tasks.
2. If the bottleneck is Python bytecode, threads do not help on the default build: the glossary defines the GIL as the mechanism that lets only one thread execute Python bytecode at a time. Use ProcessPoolExecutor (or multiprocessing), or a free-threaded build if every dependency supports it.
3. If the bottleneck is native code that releases the GIL, threads give parallelism without the serialisation cost of processes; confirm with a two-workers-versus-one run on real data.
4. For processes, set the start method explicitly with get_context(). The documentation states that on POSIX the default changed from fork to forkserver in Python 3.14 and that macOS has defaulted to spawn since 3.8; arguments and results must be picklable, so pass identifiers rather than large objects and open connections inside the worker's initializer.
5. Bound everything: max_workers, an asyncio.Semaphore, or a queue size; unbounded fan-out moves the failure to the downstream service.
6. Combine models on purpose: asyncio.to_thread for a blocking call inside a loop, loop.run_in_executor with a process pool for CPU work inside an async server.
7. Write the shutdown path: executor.shutdown(cancel_futures=True), task cancellation, and a timeout on every future.result().

Expected result

A short decision note naming the bottleneck, the model, the bound and the start method, plus a small benchmark showing the chosen model beating the single-threaded baseline on the real workload.

Limits and test basis

The rules follow the cited documentation, not measurements. Mixed workloads may need two pools. Free-threaded builds change step 2; see the GIL article and the open question on when such builds pay off.