Inter-process communication

Pipes and FIFOs, signals and their handlers, shared memory with semaphores, message queues, and local sockets as the general-purpose option.

The mechanisms and when each fits

import os, socket, struct

# pipe between related processes
r, w = os.pipe()
pid = os.fork()
if pid == 0:
    os.close(r)
    os.write(w, b"from the child\n")
    os._exit(0)
os.close(w)
print(os.read(r, 1024))
os.waitpid(pid, 0)

# Unix domain socket: bidirectional, no network stack involved
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
srv.bind("/tmp/app.sock")
srv.listen(8)
conn, _ = srv.accept()
conn.sendall(struct.pack("<I", 42) + b"payload")

Shared memory and its cost

from multiprocessing import shared_memory, Semaphore
import numpy as np, time

shm = shared_memory.SharedMemory(create=True, size=1024 * 1024)
buf = np.ndarray((1024 * 1024 // 8,), dtype=np.int64, buffer=shm.buf)

sem = Semaphore(1)                 # a semaphore is required; memory alone is not enough

with sem:
    buf[0] = 12345                 # no serialisation, no kernel copy
time.sleep(0.01)

assert buf[0] == 12345
shm.close()
shm.unlink()

• Shared memory avoids the copy that a socket imposes, which is why it is the fastest option for large payloads.
• It also gives you no synchronisation, no framing and no lifetimes — a semaphore and a documented layout are your responsibility.
• A crashed writer can leave the region in an inconsistent state; a checksum or a generation counter makes that detectable.
• Names and permissions are global to the machine, so a key collision between unrelated applications is possible.
• The kernel copies data twice for a socket or pipe: from writer to kernel buffer, then kernel to reader.

Signals are notifications, not messages

• Signals carry no payload beyond the signal number, and standard signals are not queued — two identical ones may collapse into one.
• A handler runs asynchronously and may interrupt any instruction, so it can only safely call async-signal-safe functions.
• Writing to a pipe is the standard way to get out of a handler and into normal code.
• SIGKILL and SIGSTOP cannot be caught, blocked or ignored.
• SIGTERM is the polite shutdown request; send it, wait, then escalate only if necessary.

import signal, os

def handler(signum, frame):
    # do NOT do real work here; set a flag or write one byte
    os.write(wake_fd, b"x")

signal.signal(signal.SIGTERM, handler)      # graceful shutdown request
signal.signal(signal.SIGHUP, handler)       # traditionally reload configuration

# never install a handler for SIGKILL or SIGSTOP; the kernel ignores the request

FAQ

Related

Concurrency models: threads, async and processes What an operating system does

Last refreshed 2026-09-18.