snaptrace_threaddestructor calls PyGILState_Ensure() after interpreter finalization, causing SIGSEGV

snaptrace_threaddestructor calls PyGILState_Ensure() after interpreter finalization, causing SIGSEGV

Summary

snaptrace_threaddestructor is registered as a pthread TLS destructor (snaptrace.c:2045). It is therefore invoked by glibc whenever a traced thread terminates — including when that happens after Py_FinalizeEx() has already torn down the interpreter. The function calls PyGILState_Ensure() unconditionally (snaptrace.c:285), which dereferences interpreter state that no longer exists, and the process dies with SIGSEGV (exit 139).

tracer.stop() and tracer.save() both return successfully first — the trace file is written correctly — so the crash looks disconnected from tracing and only shows up as a non-zero exit code after all real work has finished.

Environment

• Linux x86_64 (Debian trixie), CPython 3.12.13
• Reproduced on VizTracer 1.1.1 (latest on PyPI) and on current master (293a7e3) — snaptrace_threaddestructor is unchanged between the two
• Reproduced in a container; nothing exotic in the environment

Reproduction

import threading
import time

from viztracer import VizTracer


def busy():
    deadline = time.time() + 2.0
    while time.time() < deadline:
        sorted({str(i): [i] for i in range(200)})


tracer = VizTracer(verbose=0, ignore_c_function=False, ignore_frozen=False)
tracer.start()

for _ in range(4):
    threading.Thread(target=busy, daemon=True).start()

time.sleep(0.3)
tracer.stop()
tracer.save("/tmp/repro.json")
print("stop() and save() returned; exiting with threads still running", flush=True)

for i in $(seq 1 30); do python repro.py >/dev/null 2>&1 || echo "crash"; done

8 of 30 runs segfault on current master (6 of 30 on 1.1.1), always after the final print() has executed. Joining the threads before exit eliminates it entirely (30/30 clean), which is consistent with the destructor being safe as long as the thread dies while the interpreter is still valid.

ignore_c_function=True does not help.

Backtrace

Captured by attaching gdb after freezing the process at the fault (a raw SIGSTOP from a SIGSEGV handler — a live ptrace-attached debugger perturbs the timing enough to hide the race):

#0  0x... in ?? ()  from libpython3.12.so.1.0     <- consistent with new_threadstate
#1  0x... in ?? ()  from libpython3.12.so.1.0     <- consistent with PyGILState_Ensure
#2  snaptrace_threaddestructor (key=0x...) at src/viztracer/modules/snaptrace.c:285
#3  snaptrace_threaddestructor (key=0x...) at src/viztracer/modules/snaptrace.c:281
#4  0x... in ?? ()  from libc.so.6                <- pthread TLS destructor sweep
#5  0x... in ?? ()  from libc.so.6
#6  0x... in ?? ()  from libc.so.6                <- thread exit

Frame #2 resolves to exactly the PyGILState_Ensure() line. Fault address is 0x60 — a small offset off a null pointer, i.e. a field read on an interpreter/thread-state struct that is already gone.

At the same moment the main thread is in interpreter/process teardown — in one capture inside _PyPathConfig_ClearGlobal (late Py_FinalizeEx), in another inside __cxa_finalize (i.e. already past finalization, running shared-library destructors during exit()).

The deterministic variant

The daemon-thread case above is a race, but there is a common real-world pattern where it is 100% reproducible: an application that embeds a C++ extension module holding a worker thread pool as a process-lifetime static, where those threads call PyGILState_Ensure() to hand results back to Python.

Such a thread is not joined when the job finishes — it is joined by the C++ static destructor chain that runs inside exit(). Because main() calls Py_RunMain() (which finalizes the interpreter) and only then calls exit(), that thread is guaranteed to die after finalization. Once VizTracer has traced it even once, the crash is not probabilistic.

We observed exactly this in a production Python 3.12 service: 6 out of 6 runs crashed with the tracer enabled, and 0 out of 6 with it disabled, everything else identical. The main thread's stack at crash time was:

__libc_start_main → exit() → [static destructors] → ~ConcurrentHashMap<…>
  → ~ThreadPoolExecutor → stopAndJoinAllThreads → std::thread::join()
    → pthread_join   [blocked, waiting on the very thread that is crashing]

Suggested fix

Skip the cleanup when the interpreter is finalizing. snaptrace.c already includes pythoncapi_compat.h, which provides a portable Py_IsFinalizing() for every supported version, so no version guard is needed:

if (info) {
    if (Py_IsFinalizing()) {
        return;
    }
    PyGILState_STATE state = PyGILState_Ensure();
    ...

The body of the destructor only does Py_CLEAR() and PyMem_FREE() on per-thread buffers, so the early return just leaks them moments before the process exits — which is what the CPython C API leaves you able to do at that point anyway.

This is the same guard that other native code commonly uses in the same position, e.g. deleters that need the GIL to drop a pybind11::object check Py_IsFinalizing() and discard rather than attempt cleanup.

I have this patched and verified locally and will open a PR:

Tracing still works normally with the patch (trace files written, ~566k events, valid JSON), and the application's own output is byte-for-byte identical to an untraced run.

Happy to adjust the approach if you'd prefer a different one.