In model serving, a common pattern is to expose an HTTP API with FastAPI and start PyTorch worker processes behind that API. FastAPI handles requests. PyTorch handles model loading and inference.

This combination is reasonable. But if torch.multiprocessing.spawn is called inside a synchronous FastAPI endpoint, there is a subtle failure mode:

The FastAPI main process stays alive, but the PyTorch workers receive SIGINT and exit.

This post walks through that issue. The root cause is not in a single layer. It comes from the interaction between several mechanisms:

FastAPI sync endpoint
  -> AnyIO worker thread
  -> torch.multiprocessing.spawn
  -> Linux PR_SET_PDEATHSIG

The discussion below mainly applies to Linux. PR_SET_PDEATHSIG is Linux-specific, so the same behavior may not reproduce on macOS or Windows.

The Symptom

Our service had an endpoint that started background workers. Internally, it called torch.multiprocessing.spawn to launch multiple worker processes.

Simplified, it looked like this:

@app.post("/start-workers")
def start_workers():
    torch_mp_spawn(worker_main, nprocs=8, join=False, start_method="spawn")
    return {"status": "started"}

After the service started, the workers were created successfully and ran normally.

But in a Kubernetes cluster, we observed this behavior: after the service had been idle for a while, all 8 workers received SIGINT almost at the same time. In Python, this showed up as KeyboardInterrupt, and the workers exited.

The obvious explanations did not fit:

  1. Nobody pressed Ctrl+C.
  2. Kubernetes did not restart the Pod.
  3. There was no OOM kill event.
  4. The FastAPI main process was still alive and could still receive HTTP requests.

If the main process had exited, it would be normal for child processes to go away. But here the main process was still running. So why did the workers receive SIGINT?

What PyTorch Spawn Does

The first clue is in torch.multiprocessing.spawn.

When PyTorch starts a child process, it sets a parent-death signal:

# torch/multiprocessing/spawn.py
_prctl_pr_set_pdeathsig(signal.SIGINT)

This maps to the Linux call prctl(PR_SET_PDEATHSIG, signal).

The rough idea is: if the parent exits, the kernel sends the configured signal to the child process. PyTorch sets this signal to SIGINT, so that worker processes do not become orphaned if the parent fails unexpectedly.

This explains why the workers received SIGINT. But it does not yet explain why the signal was triggered while the FastAPI main process was still alive.

Parent Does Not Always Mean Process

The key detail is in the Linux semantics of PR_SET_PDEATHSIG.

The Linux man page includes this warning:

the “parent” in this case is considered to be the thread that created this process.

In other words, the parent here is not necessarily the whole parent process. It is the parent thread that created the child process.

So the question becomes:

Which thread called torch.multiprocessing.spawn?

If that thread exits, Linux may consider the worker’s parent to be gone, even if the FastAPI main process is still alive. The kernel can then send SIGINT to the worker.

FastAPI Sync Endpoints

The endpoint was originally a synchronous function:

@app.post("/start-workers")
def start_workers():
    torch_mp_spawn(worker_main, nprocs=8, join=False, start_method="spawn")
    return {"status": "started"}

In FastAPI, an async def endpoint runs in the event loop. A regular def endpoint is different: Starlette/AnyIO runs it in a worker thread so that it does not block the event loop. I wrote a separate source walk-through for this part in Tracing FastAPI def Endpoints into AnyIO Worker Threads.

So the real call path was not:

FastAPI main thread -> torch.multiprocessing.spawn

It was:

FastAPI request
  -> AnyIO worker thread
  -> torch.multiprocessing.spawn
  -> PyTorch worker process

From the Linux kernel’s point of view, the parent thread of the PyTorch workers was the AnyIO worker thread that called spawn.

That explains the core of the issue: if that AnyIO worker thread is later reclaimed, the PyTorch workers receive SIGINT. Whether the FastAPI main process is still alive is not the deciding factor.

Why It Did Not Happen Immediately

There is one more detail: the workers did not exit immediately after the endpoint returned. Usually, the service would sit idle for a while, and then another request would trigger the workers to exit.

This comes from the thread pool behavior.

In the environment where we saw the issue, the worker thread used for the synchronous endpoint did not disappear immediately after the request finished. It became idle. Later, when the thread pool was used again, AnyIO cleaned up idle workers and reclaimed threads that had been idle long enough.

The sequence was:

/start-workers request arrives
  -> AnyIO worker thread A runs start_workers()
  -> thread A calls torch.multiprocessing.spawn
  -> PyTorch workers bind PDEATHSIG to thread A
  -> /start-workers returns, thread A becomes idle
  -> later, a new request triggers idle worker cleanup
  -> thread A is reclaimed
  -> Linux sends SIGINT to the PyTorch workers

This is why the symptom looked like a delayed failure after the service had been idle.

The important point is not a specific timeout value. The exact behavior can vary across Python, AnyIO, Starlette, FastAPI, and PyTorch versions. The important condition is: the thread that created the PyTorch workers is later reclaimed.

A Minimal Reproduction

The issue can be reproduced with a small program. It does not need to load a real model. It only needs to start PyTorch workers from a synchronous endpoint and keep those workers alive.

import os
import time

import uvicorn
from fastapi import FastAPI
from torch.multiprocessing.spawn import spawn as torch_mp_spawn

app = FastAPI()


def worker_main(i):
    pid = os.getpid()
    print(f"[worker {i}] started, pid={pid}", flush=True)

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        print(f"[worker {i}] received SIGINT, pid={pid}", flush=True)
        raise


@app.post("/start-workers")
def start_workers():
    torch_mp_spawn(worker_main, nprocs=8, join=False, start_method="spawn")
    return {"status": "started"}


@app.post("/spin")
def spin():
    time.sleep(1)
    return {"status": "spun"}


if __name__ == "__main__":
    uvicorn.run(app, host="127.0.0.1", port=8100)

Start the service:

python demo.py

Then send a few requests:

curl -X POST http://127.0.0.1:8100/start-workers &
curl -X POST http://127.0.0.1:8100/spin &
curl -X POST http://127.0.0.1:8100/spin

sleep 15

curl -X POST http://127.0.0.1:8100/spin

If the issue is reproduced, the workers created by /start-workers will print that they received SIGINT.

The exact timing and trigger can vary by version. The goal of the reproduction is not to prove that sleep 15 is special. It is to show that once the parent thread of the workers is reclaimed, PDEATHSIG can be triggered.

The Fix

The principle is:

Do not attach the lifecycle of long-running PyTorch workers to a temporary worker thread from the web framework’s default thread pool.

One direct fix is to make the endpoint async def and submit the actual worker-starting logic to an executor that we own and keep alive:

import asyncio
import concurrent.futures

worker_executor = concurrent.futures.ThreadPoolExecutor(
    max_workers=1,
    thread_name_prefix="worker_launcher",
)


@app.post("/start-workers")
async def start_workers():
    def _start():
        torch_mp_spawn(worker_main, nprocs=8, join=False, start_method="spawn")

    await asyncio.get_running_loop().run_in_executor(worker_executor, _start)
    return {"status": "started"}

The important part is not async def by itself. The important part is that spawn is no longer executed by FastAPI’s default thread pool for synchronous endpoints. worker_executor is a process-level object that we own, so it is not destroyed just because one request ends and the default thread pool prunes idle workers.

In a more complete production design, the boundary can be made clearer:

  1. Use a dedicated supervisor thread to manage model workers.
  2. Use a separate supervisor process to manage PyTorch workers.
  3. Let the web endpoint only submit control commands, and let a long-running background component perform the actual worker startup.

Model workers, GPU processes, and background supervisors all have their own lifecycle. They usually should not be tied directly to the execution thread of an HTTP request handler.

Takeaways

The core issue is not complicated, but it crosses several abstraction layers:

FastAPI sync endpoint
  -> AnyIO worker thread
  -> torch.multiprocessing.spawn
  -> Linux PR_SET_PDEATHSIG
  -> parent thread is reclaimed
  -> PyTorch workers receive SIGINT

When we see KeyboardInterrupt, it is natural to look for Ctrl+C, a Pod restart, or an OOM event. In this case, the signal was actually sent by the Linux kernel.

The lesson I took from this is: when a model service uses a web framework, a thread pool, and multiprocessing together, the lifecycle ownership of long-running resources needs to be explicit.

On Linux, one detail is especially important: the parent in PR_SET_PDEATHSIG means the parent thread that created the child process, not necessarily the whole parent process.

References