[BUG] `Kubernetes discovery GetConns can leak mutex lock during concurrent cache initialization`

Author: buvidk1234Created Jul 8, 2026Updated Jul 8, 2026
Labelsbug

OpenIM Server Version

main

Operating System and CPU Architecture

Linux (AMD)

Deployment Method

Source Code Deployment

Bug Description and Steps to Reproduce

KubernetesConnManager.GetConns uses a double-check pattern when connMap[serviceName] is not found on the first read-lock check.

If another goroutine initializes the same service connections before the second check runs, the second check returns cached connections directly while still holding k.mu.Lock().

go
k.mu.Lock()
// Check if another goroutine has already initialized the connections when we released the read lock
conns, exists = k.connMap[serviceName]
if exists {
    return conns, nil
}
k.mu.Unlock()

This leaves the mutex locked indefinitely. Later calls that need k.mu can block forever, including GetConns, endpoint refresh handling, AddOption, or Close.

This can affect Kubernetes deployments under concurrent startup or traffic spikes, especially gateway fanout / online push paths that call GetConns for message gateway services.

Expected behavior

GetConns should always release k.mu before returning from the second cache-check branch.

Actual behavior

The second cache-hit branch returns without unlocking k.mu.

Possible trigger scenario

  1. Goroutine A calls GetConns(serviceName) and misses the first read-lock check.
  2. Goroutine B also calls GetConns(serviceName) and misses the first read-lock check.
  3. Goroutine A initializes and stores connMap[serviceName].
  4. Goroutine B enters the second check, sees the initialized cache, and returns without unlocking.
  5. Future discovery operations block on k.mu.

Suggested fix

Unlock before returning from the second cache-hit branch:

go
if exists {
    k.mu.Unlock()
    return conns, nil
}

Screenshots Link

No response

Source: openimsdk/open-im-server