[BUG] `Kubernetes discovery GetConns can leak mutex lock during concurrent cache initialization`
OpenIM Server Version
main
Operating System and CPU Architecture
Linux (AMD)
Deployment Method
Source Code Deployment
Bug Description and Steps to Reproduce
KubernetesConnManager.GetConns uses a double-check pattern when connMap[serviceName] is not found on the first read-lock check.
If another goroutine initializes the same service connections before the second check runs, the second check returns cached connections directly while still holding k.mu.Lock().
k.mu.Lock()
// Check if another goroutine has already initialized the connections when we released the read lock
conns, exists = k.connMap[serviceName]
if exists {
return conns, nil
}
k.mu.Unlock()This leaves the mutex locked indefinitely. Later calls that need k.mu can block forever, including GetConns, endpoint refresh handling, AddOption, or Close.
This can affect Kubernetes deployments under concurrent startup or traffic spikes, especially gateway fanout / online push paths that call GetConns for message gateway services.
Expected behavior
GetConns should always release k.mu before returning from the second cache-check branch.
Actual behavior
The second cache-hit branch returns without unlocking k.mu.
Possible trigger scenario
- Goroutine A calls
GetConns(serviceName)and misses the first read-lock check. - Goroutine B also calls
GetConns(serviceName)and misses the first read-lock check. - Goroutine A initializes and stores
connMap[serviceName]. - Goroutine B enters the second check, sees the initialized cache, and returns without unlocking.
- Future discovery operations block on
k.mu.
Suggested fix
Unlock before returning from the second cache-hit branch:
if exists {
k.mu.Unlock()
return conns, nil
}Screenshots Link
No response
Source: openimsdk/open-im-server