TopicLookup memory leak
We are using SignalR version 2.4.2.
We are facing an incremental memory usage of each node of our SignalR application, typically it cause an OOM error and crash. We can't find if there is any object retained. From a dump we are seeing lot of RedisMessageBus-TopicLookup
The clients are some distributed cpp applications used to send messages to specific connected device. On server SignalR we have 6 balanced nodes, sticky session and redis as backplane.
Startup.cs
public void Configuration(IAppBuilder appBuilder)
{
...
appBuilder.UseNinjectMiddleware(CreateKernel);
}
private static IKernel CreateKernel()
{
Kernel.Bind<IBroadcastManager>().To<BroadcastManager>().InSingletonScope();
Kernel.Bind<IHubContext>().ToMethod(context =>
GlobalHost.ConnectionManager.GetHubContext<BrokerHubManager>()
).WhenInjectedInto<IBroadcastManager>();
...
return Kernel;
}Program.cs
class Program
{
static void Main(string[] args)
{
...
}
}
internal class SignalRService : ServiceControl
{
private IBroadcastManager _broadcaster;
public SignalRService()
{
Task.Run(() =>
{
WebApp.Start<Startup>(Address);
_broadcaster = Startup.Kernel.Get<IBroadcastManager>();
...
}).Wait();
_timer = new Timer(MyTimerAsync, null, 0, Convert.ToInt32(Startup.LostDeviceTimer.TotalMilliseconds));
}
private async void MyTimerAsync(object stateInfo)
{
_broadcaster.XXX();
...
}
}BrokerHubManager.cs
public class BrokerHubManager : Hub
{
public BrokerHubManager()
{
...
}
}BroadcastManager.cs
public class BroadcastManager : IBroadcastManager
{
private IHubContext _context;
private BroadcastManager(IHubContext context)
{
_context = context;
}
}We have a timer that each 15 minutes takes all the not alive devices and disconnect them:
private async void HeartbeatDevicesAsync(object stateInfo)
{
try
{
if (Startup.TraceMode) BrokerLogger.LogInfo("Timer Heartbeat device - Checking connections ...");
var heartBeat = GlobalHost.DependencyResolver.Resolve<ITransportHeartbeat>();
//SignalR connections
var connectionList = heartBeat?.GetConnections();
foreach (var connection in connectionList)
{
if (!connection.IsAlive)
{
if (Startup.TraceMode) BrokerLogger.LogWarn($"SignalR Graceful disconnection Id -> {connection.ConnectionId}");
await connection.Disconnect();
}
}
}
catch (Exception e)
{
if (Startup.TraceMode) BrokerLogger.LogErr(e.ToString());
}
if (Startup.TraceMode) BrokerLogger.LogInfo($"Timer Heartbeat is sleeping for {Startup.HeartbeatDevicesTimer.TotalSeconds} seconds ...");
}Expected behavior
We need that memory goes up and down accordingly to the live connestions, actually is growing day by day.
Actual behavior
Here is some informations took from DotMemory when the applications "starts to grow"
Source: SignalR/SignalR