Memory leak : ReddisClient () slowly increases the memory and finally crashes the server
I'm using "redis": "^3.1.2". My containers are restarting/ crashing in prod due to a memory leak. The traffic is very very less around 650 in 24 hours. Following are my observations:
For the first few 100 requests the memory would be around 200/300 MB and it would stay the same even after an hour without traffic.GC couldn't clear the memory. so after subsequent few 100 requests, the memory increases and it crashes my container (my container setting in prod is 1GB.)
I have used google chrome inspect to understand which object or function is causing this. And it turned out to be Redis client is holding very less memory at the beginning and after leaving it for a while the memory it holds keeps on increasing.(I have taken the snapshots of heap over time and compared the difference).
I'm actually connecting to reddis for every request and closing the connection using the connection.quit(). But another observation I found inside the Redis module there is a stream that continuously runs and does something recursively. upon debugging I found the connection is still alive. So I changed the method to end() instead of quit and it still produces the stream. and the connection is still alive. At this point, I'm literally stuck. please help me with the following question
I) is it advised to connect to Redis and close for every request.
- Is it normal to have a memory leak when using Redis module
- irrespective of quit or end the stream doesn't stop in the background (for example keep a debug in individual.commands.js -> info_callback method and monitor the connection status.)
Code
`connect(config) { const connectionRetryTime = 1000 * 60 * 60; const clientOptions = { host: config.endpoint.host, port: config.endpoint.port, socket_keepalive: true, password: config.redisAuth, tls: true, retry_strategy: (options) => { if (options.error && options.error.code === 'ECONNREFUSED') { return new Error('The server refused the connection'); } if (options.total_retry_time > connectionRetryTime) { return new Error('Retry time exhausted'); } if (options.attempt > 2) { return undefined; }
return Math.min(options.attempt * 100, 3000);
},
};
return new Promise(((resolve, reject) => {
try {
this.cacheProvider = redis.createClient(clientOptions);
this.cacheProvider.on('error', () => {
// eslint-disable-next-line no-console
console.error('Error Connecting to Redis error');
});
// Get promisified version of RedisClient functions
this.getAsync = promisify(this.cacheProvider.get).bind(this.cacheProvider);
this.setAsync = promisify(this.cacheProvider.set).bind(this.cacheProvider);
this.setexAsync = promisify(this.cacheProvider.setex).bind(this.cacheProvider);
this.keysAsync = promisify(this.cacheProvider.keys).bind(this.cacheProvider);
this.pingAsync = promisify(this.cacheProvider.ping).bind(this.cacheProvider);
this.quitAsync = promisify(this.cacheProvider.quit).bind(this.cacheProvider);
return resolve(this.cacheProvider);
} catch (error) {
return reject(error);
}
}));
}
init(config) { return this.connect(config); }
healthCheck() { return this.pingAsync(); }
async release() { return this.cacheProvider.quitAsync; } `
For each request I'm
init(); get(); release();
Environment:
- **Node.js Version14:
- Redis Server Version:
- **Node Redis Version3.1.2:
- **PlatformmacOS 12.4:
Source: redis/node-redis