Concurrency models: threads, async and processes

Thread pools versus event loops, what async actually does under the hood, multi-process workers, and how to pick a model that matches the workload.

Four models and what they share

Model	Parallel CPU	Handles many connections	Isolation	Typical stack
Thread per request	Yes	Poorly, one thread each	Shared memory	Java servlets, classic Rails
Thread pool	Yes	Bounded by pool size	Shared memory	Most web frameworks
Event loop with async	Single core per loop	Very well	Shared memory	Node.js, Python asyncio, nginx
Worker processes	Yes	Bounded by worker count	Strong, separate memory	PHP-FPM, gunicorn, Celery
Actor or CSP	Yes	Well	Isolated mailboxes	Erlang, Go channels, Akka

Every model is one of two strategies for the same problem: block and let the scheduler manage the wait, or never block and manage the wait yourself with callbacks and state machines.

What async really does

import asyncio

async def fetch(name, seconds):
    await asyncio.sleep(seconds)      # yields control, the loop runs others
    return name

async def main():
    # concurrent, not parallel: all on one thread
    results = await asyncio.gather(
        fetch("a", 0.3), fetch("b", 0.1), fetch("c", 0.2)
    )
    print(results)

    # bound the concurrency so a dependency is not overwhelmed
    gate = asyncio.Semaphore(10)

    async def limited(item):
        async with gate:
            return await fetch(item, 0.05)

    await asyncio.gather(*(limited(i) for i in range(200)))

asyncio.run(main())

An event loop is concurrent but not parallel: one CPU-bound coroutine blocks every other task on that loop.
Blocking calls inside async code defeat the model. A synchronous database driver in an async handler stalls the whole loop.
CPU-bound work belongs in a process pool or a separate service, not in the same loop as latency-sensitive I/O.
Run one loop per core and let the operating system schedule them, rather than trying to parallelise inside one loop.
Cancellation and timeouts are part of correctness: a task with no timeout can hold a resource forever.

Choosing for a workload

Workload	Model	Reason
Many slow external calls	Async event loop	Thousands of concurrent waits, no CPU cost
Heavy CPU per request	Worker processes	Bypasses the interpreter lock and isolates crashes
Mixed I/O and CPU	Async for I/O, pool for CPU	Keeps the loop free while fanning out compute
Long-lived streaming connections	Async or an event-driven server	Connection count, not throughput, is the constraint
Batch job with stages	Process pool with a queue	Backpressure and failure isolation
Untrusted plugin execution	Separate process	Memory isolation and a kill path

# check what a worker process is actually doing
ps -eLf | grep gunicorn | head          # threads per process
ss -tan state established | wc -l       # connections held open
top -H -p 12345                         # per-thread CPU inside one process

💡

The model is a resource-budget decision. Threads cost memory per stack and context-switch overhead; async costs complexity in the code. Pick whichever cost you can pay and measure it, rather than picking by preference.

FAQ

Why did my async service use only one core?

A single event loop runs on one thread. Start several processes each with its own loop, and let the load balancer spread connections across them.

Do processes really give better isolation?

Yes — crashes and memory corruption do not propagate, and a process can be killed precisely. The cost is a heavier startup and explicit data transfer between processes.

Inter-process communication Races, deadlocks and starvation

Last refreshed 2026-09-18.

Concurrency models: threads, async and processes

Four models and what they share

What async really does

Choosing for a workload

FAQ

Related