[1.4] Janus crashes with general protection fault in kernel log

Author: bijaykandel37Created Jul 13, 2026Updated Sep 1, 2026
Labelsmultistream

janus_videoroom_helper_thread() in plugins/janus_videoroom.c crashes with SIGSEGV inside strlen() (called via JANUS_LOGjanus_vprintfg_vasprintfvasprintf) when logging its own exit message:

JANUS_LOG(LOG_VERB, "[%s/#%d] Leaving VideoRoom helper thread\n", room->room_id_str, helper->id);

This happens at plugins/janus_videoroom.c:14261. Reproduced twice independently on production, one day apart, with identical signatures — in both cases two helper threads crash simultaneously at the exact same instruction, strongly suggesting the room (and its room_id_str) is freed by the room-destroy path while one or more helper threads for that room are still exiting their loop and about to log their own shutdown message.

Environment

  • Janus version: 1.4.2 (build 1402)
  • Commit: 94408ffccbc1a7c39dac82d55e5dd475c807e770
  • OS: Ubuntu (24.04-based), kernel via AWS EC2
  • Compiled: Mon May 11 08:58:51 UTC 2026
  • Plugins active: VideoRoom, AudioBridge, Streaming, SIP, NoSIP, VideoCall, TextRoom, RecordPlay, EchoTest
  • Transports active: HTTP, WebSockets

Steps to reproduce

Not deterministically reproducible on demand — occurs under normal production load with routine room creation/destruction and multiple active VideoRoom helper threads (multi-forwarder / simulcast setups spawn helper threads per room). Appears tied to a room being destroyed while its helper thread(s) are still in their processing loop.

Crash 1 — backtrace (gdb, via apport core, Jul 12 2026)

#0  __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:76
#1  __printf_buffer (format="%s%s%s%s[%s/#%d] Leaving VideoRoom helper thread\n", ...) at ./stdio-common/vfprintf-process-arg.c:443
#2  __vasprintf_internal () at ./libio/vasprintf.c:102
#3  g_vasprintf ()
#4  janus_vprintf (format=...) at log.c:149
    str = 0x0
#5  janus_videoroom_helper_thread (data=0x74b0fc002fb0) at plugins/janus_videoroom.c:14261
#6  start_thread ()
#7  __clone3 ()

A second thread (different LWP) crashed at the identical instant, same backtrace, same line — both helper threads for the same room exiting concurrently.

Crash 2 — backtrace (gdb via coredumpctl, Jul 13 2026, ~24h later)

Identical signature and line number (plugins/janus_videoroom.c:14261), again with two threads crashing simultaneously at the same instruction (LWP 1079909 and 1079910 in this instance). str = 0x0 again in janus_vprintf's frame.

Relevant source (plugins/janus_videoroom.c, lines ~14234–14261)

static void *janus_videoroom_helper_thread(void *data) {
    janus_videoroom_helper *helper = (janus_videoroom_helper *)data;
    janus_videoroom *room = helper->room;
    janus_videoroom_publisher_stream *ps = NULL;
    GList *subscribers = NULL;
    JANUS_LOG(LOG_VERB, "[%s/#%d] Joining VideoRoom helper thread\n", room->room_id_str, helper->id);
    janus_videoroom_rtp_relay_packet *pkt = NULL;
    while(!g_atomic_int_get(&stopping) && !g_atomic_int_get(&room->destroyed) && !g_atomic_int_get(&helper->destroyed)) {
        pkt = g_async_queue_pop(helper->queued_packets);
        if(pkt == &exit_packet)
            break;
        janus_mutex_lock(&helper->mutex);
        ps = pkt->source;
        subscribers = g_hash_table_lookup(helper->subscribers, ps);
        if(subscribers != NULL) {
            g_list_foreach(subscribers,
                pkt->is_rtp ? janus_videoroom_relay_rtp_packet : janus_videoroom_relay_data_packet,
                pkt);
        }
        janus_mutex_unlock(&helper->mutex);
        janus_videoroom_rtp_relay_packet_free(pkt);
    }
    JANUS_LOG(LOG_VERB, "[%s/#%d] Leaving VideoRoom helper thread\n", room->room_id_str, helper->id);
    janus_refcount_decrease(&helper->ref);
    janus_refcount_decrease(&room->ref);
    g_thread_unref(g_thread_self());
    return NULL;
}

Suspected root cause

The loop condition checks room->destroyed, but room->room_id_str appears to be freed by the room-destruction path before janus_refcount_decrease(&room->ref) is called at the bottom of this function — i.e. before the helper thread has fully released its hold on the room. If the room's destroy callback frees room_id_str as soon as room->destroyed is set (rather than waiting for the refcount to reach zero), any helper thread that is mid-loop or just breaking out of it will read a dangling pointer when it logs its own exit message, causing strlen(NULL) / strlen(<freed memory>) inside the printf formatting machinery.

The fact that two helper threads crashed at the identical instant in both independent occurrences supports this: multiple helpers belonging to the same room hit their shutdown log line concurrently right as the room is torn down.

Suggested fix

Cache the room ID as a local string at the top of janus_videoroom_helper_thread, before the loop, so neither the "Joining" nor "Leaving" log lines dereference room->room_id_str after the room may have started being destroyed:

char room_id_str[64];
g_snprintf(room_id_str, sizeof(room_id_str), "%s", room->room_id_str);
...
JANUS_LOG(LOG_VERB, "[%s/#%d] Joining VideoRoom helper thread\n", room_id_str, helper->id);
...
JANUS_LOG(LOG_VERB, "[%s/#%d] Leaving VideoRoom helper thread\n", room_id_str, helper->id);

Alternatively, ensure room_id_str (and any other fields referenced during helper shutdown) are only freed once all helper threads' refcounts have actually dropped to zero, not merely when room->destroyed is set.

Additional context

Two independent crash captures, ~24 hours apart, both production, both identical signature/line — this is a real, reproducible race rather than an isolated fluke. Happy to provide the full thread apply all bt output from either core if useful.