Ad – 728×90
🚀 Advanced Python

Python Multiprocessing – True Parallelism for CPU-Bound Tasks

Python multiprocessing bypasses the GIL by running separate Python processes. Each process has its own memory space and Python interpreter, enabling true CPU parallelism. Use multiprocessing when you have CPU-intensive work: number crunching, image processing, machine learning training.

⏱️ 22 min read🎯 Advanced📅 Updated 2026

The Process Class

multiprocessing.Process spawns a new OS process.

Python
from multiprocessing import Process
import os

def worker(name):
    print(f"Worker {name} | PID: {os.getpid()}")

if __name__ == "__main__":  # Required on Windows!
    processes = []
    for i in range(3):
        p = Process(target=worker, args=(i,))
        processes.append(p)
        p.start()

    for p in processes:
        p.join()
▶ Output
Worker 0 | PID: 12345 Worker 1 | PID: 12346 Worker 2 | PID: 12347
💡
Tip

Always put multiprocessing code inside if __name__ == "__main__": to prevent recursive spawning on Windows.

Pool – Parallel Map Operations

Pool distributes work across multiple CPU cores automatically.

Python
from multiprocessing import Pool
import time

def square(n):
    time.sleep(0.1)   # Simulate work
    return n * n

if __name__ == "__main__":
    numbers = list(range(8))

    # Sequential: 0.8s
    # With 4 workers: ~0.2s
    with Pool(processes=4) as pool:
        results = pool.map(square, numbers)

    print(results)
▶ Output
[0, 1, 4, 9, 16, 25, 36, 49]

ProcessPoolExecutor – Modern API

concurrent.futures.ProcessPoolExecutor is the high-level interface for multiprocessing.

Python
from concurrent.futures import ProcessPoolExecutor

def heavy_computation(n):
    return sum(i**2 for i in range(n))

if __name__ == "__main__":
    inputs = [1_000_000, 2_000_000, 3_000_000]

    with ProcessPoolExecutor(max_workers=3) as exe:
        results = list(exe.map(heavy_computation, inputs))

    print(results)
▶ Output
[333332833333500000, 2666666000000, 9000001500000]
Ad – 336×280

Multiprocessing vs Threading

Choose the right tool for the job.

Python
# I/O bound (waiting for network, disk, DB)
# → Use threading or asyncio
# → Threads share memory, low overhead

# CPU bound (calculation, image processing)
# → Use multiprocessing
# → Each process gets full CPU core
# → Bypasses the GIL

# Rule of thumb:
import os
CPU_COUNT = os.cpu_count()
print(f"Available cores: {CPU_COUNT}")
Ad – 728×90