[BUG] 在 fork 后, AOF 重写子进程仍然被 jemalloc mutex 锁定 (Redis 7.4.7)
**Describe the bug** A background AOF rewrite child process remained blocked for approximately 7 days and 22 hours on a production Redis 7.4.7 instance. Redis continued serving requests, but the AOF rewrite never completed. The child process was blocked in a jemalloc mutex: PID 426998 PPID 2096105 STAT S futex_wait_queue Redis persistence state: aof_rewrite_in_progress:1 aof_rewrite_scheduled:0 aof_current_rewrite_time_sec:686958 aof_last_bgrewrite_status:ok aof_current_size:45757506678 aof_base_size:2226314945 The GDB backtrace was: #0 __lll_lock_wait () from /lib64/libc.so.6 #1 pthread_mutex_lock@@GLIBC_2.2.5 () from /lib64/libc.so.6 #2 malloc_mutex_lock_final () at include/jemalloc/internal/mutex.h:151 #3 je_malloc_mutex_lock_slow () at src/mutex.c:90 #4 malloc_mutex_lock () #5 je_extent_record () at src/extent.c:921 #6 je_ecache_dalloc () at src/extent.c:147 #7 pac_dalloc_impl () at src/pac.c:277 #8 je_arena_slab_dalloc () at src/arena.c:570 #9 tcache_bin_flush_impl () at src/tcache.c:477 #10 tcache_bin_flush_bottom () #11 je_tcache_bin_flush_small () #12 tcache_gc_small () #13 tcache_event () #14 je_tcache_gc_event_handler () #15 je_te_event_trigger () #16 te_event_advance () #17 thread_alloc_event () #18 imalloc_body () at src/jemalloc.c:2585 #19 imalloc () #20 je_malloc_default () #21 ztrymalloc_usable_internal () at zmalloc.c:96 #22 zmalloc () at zmalloc.c:125 #23 rdbSaveLzfStringObject () at rdb.c:363 #24 rdbSaveRawString () at rdb.c:450 #25 rdbSaveObject () at rdb.c:855 #26 rdbSaveKeyValuePair () at rdb.c:1223 #27 rdbSaveDb () at rdb.c:1416 #28 rdbSaveRio () at rdb.c:1471 #29 rewriteAppendOnlyFile () at aof.c:2381 #30 rewriteAppendOnlyFileBackground () at aof.c:2480 This appears consistent with the forked child waiting for an allocator mutex. However, I cannot confirm whether the root cause is Redis, jemalloc, or their interaction. Environment: Redis: 7.4.7 OS: TencentOS Kernel: 6.6.117-45.7.3.tl4.x86_64 Architecture: x86_64 Allocator: jemalloc-5.3.0 jemalloc-bg-thread: yes io-threads: 1 Redis modules: none Relevant configuration: appendonly yes appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 5368709120 aof-rewrite-incremental-fsync yes **To reproduce** The issue is rare and cannot currently be reproduced on demand. It occurred after long-term production operation during an automatic AOF rewrite. The AOF files were approximately: Base RDB: 2.1 GB Incremental AOF: 3.0 GB Current AOF: approximately 40 GB The rewrite child stopped making progress. Its I/O counters were: read_bytes: 0 write_bytes: 163840 There was sufficient disk space, no swap usage, no OOM event, no filesystem error, and no apparent disk saturation. **Expected behavior** The background AOF rewrite should either complete or exit with an error so Redis can report the
内容来源: redis/redis