SIGPROF backtrace deadlocks during exception unwinding

We encountered a deadlock in a FoundationDB 7.4.6 release build. The affected fdbserver remained alive but stopped processing requests, and cluster recovery stalled waiting for its TLog.

Environment:

• Linux x86_64, Ubuntu, kernel 5.15.0-161-generic
• FoundationDB 7.4.6

We inspected the live process using GDB and found:

1. Connection::read() encountered a socket EOF and threw connection_failed() (error code 1026). We verified both the exception object and the underlying Boost.Asio error (misc_category, value 2).

2. During exception unwinding, the main thread entered _Unwind_Find_FDE() in libgcc_s.so.1.

3. The thread was interrupted by the run-loop profiler’s SIGPROF handler, which called libc backtrace().

4. backtrace() re-entered _Unwind_Find_FDE() and blocked on a mutex already owned by the same main thread.

Relevant stack frames, with unresolved FDB frames omitted:

  pthread_mutex_lock
  _Unwind_Find_FDE
  ...
  _Unwind_Backtrace
  __GI___backtrace
  <FDB profiler handler>
  <signal handler called>
  ...
  _Unwind_Find_FDE
  ...
  _Unwind_RaiseException
  <FDB exception-throwing path>

The mutex state confirmed a self-deadlock:

Main thread TID: 1295175

  __lock  = 2
  __owner = 1295175
  __kind  = 0

The process also had an fdb-loopprofile thread. Inspection of the running binary confirmed calls to pthread_kill(mainThread, SIGPROF) and a configured run-loop profiling interval of 0.125 seconds.

Related reports and changes:

• https://github.com/apple/foundationdb/pull/9953
• https://github.com/apple/foundationdb/pull/1792
• https://github.com/apple/foundationdb/pull/7374
• https://forums.foundationdb.org/t/deadlock-caused-by-profiler-thread-in-storage-server/3921

PR #9953 describes the same deadlock mechanism, but its fix specifically addresses sanitizer builds. Our build does not use sanitizers.