[BUG] [Loki] Loki hangs on labels request
Author: zs-dimaCreated Oct 22, 2022Updated Sep 16, 2026
Labelscomponent/querierneed-investigationtype/bug
Often Loki hangs on labels request. However sometimes after restart it working correctly.
I tried latest, 2.5.0 и main grafana/loki versions - issue still there.
Server Oracle Cloud A1 ARM with Ubuntu 22.04.1 LTS Docker version 20.10.20, build 9fdeb9c
$ curl http://127.0.0.1:3100/ready
ready
$ curl http://127.0.0.1:3100/loki/api/v1/labels
<- infinite hangsloki.yml https://gist.github.com/zs-dima/e23237f7c7e1ff6011588fb77670412f
System is not overloaded as well
top - 11:00:50 up 1:18, 1 user, load average: 0.07, 0.04, 0.01
Tasks: 177 total, 1 running, 176 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.5 us, 0.7 sy, 0.0 ni, 98.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 11940.3 total, 9997.1 free, 554.8 used, 1388.4 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 11185.9 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1090 root 20 0 1499876 155628 43832 S 0.7 1.3 0:23.61 dockerd
1170 root 20 0 769844 49120 38308 S 0.7 0.4 0:13.55 docker-driver
2296 10001 20 0 758176 59764 40592 S 0.3 0.5 0:06.19 loki
2613 root 20 0 1792028 77404 46788 S 0.3 0.6 0:16.96 promtail
1 root 20 0 102456 11660 7724 S 0.0 0.1 0:03.22 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gpLoki logs have no errors or warnings.
however rarely "error contacting scheduler" appearing:
caller=scheduler_processor.go:87 msg="error contacting scheduler" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 10.10.0.4:9095: i/o timeout\"" addr=10.10.0.4:9095
Source: grafana/loki