Baike.dev
Log in
Back to tool/Back to issues
#4661·crowdsec

Panic in MetricsProvider.sendMetrics (lpmetrics.go:389) - nil pointer dereference

Author: ngfblogCreated Sep 5, 2026Updated Sep 14, 2026
Labelskind/bugneeds/triageos/freebsdversion/1.7.8

What happened?

CrowdSec crashes with a panic in the LP metrics background goroutine (MetricsProvider.sendMetrics), introduced by the lp metrics feature (PR #2870 / #2858). Recurred multiple times over the past week. Full panic trace below:

error: runtime error: invalid memory address or nil pointer dereference version: v1.7.8-6322745 BuildDate: 2026-06-17_18:09:57 GoVersion: 1.26.4 Platform: freebsd goroutine 182 [running]: runtime/debug.Stack() runtime/debug/stack.go:26 +0x5e github.com/crowdsecurity/go-cs-lib/trace.(*traceKeeper).writeStackTrace(0x4bd8da0, {0x4ecfc0, 0x4baacf0}) github.com/crowdsecurity/[email protected]/trace/trace.go:165 +0x16e github.com/crowdsecurity/go-cs-lib/trace.(*traceKeeper).handlePanic(0x4bd8da0, {0x0?, 0x4baa2e0?}, {0x4ecfc0, 0x4baacf0}) github.com/crowdsecurity/[email protected]/trace/trace.go:180 +0x12f github.com/crowdsecurity/go-cs-lib/trace.ReportPanic() github.com/crowdsecurity/[email protected]/trace/trace.go:48 +0x31 panic({0x4ecfc0?, 0x4baacf0?}) runtime/panic.go:860 +0x13a main.(*MetricsProvider).sendMetrics(0x871ea4200, {0xa916d0?, 0x871f232c0?}, 0x871b343c0) github.com/crowdsecurity/crowdsec/cmd/crowdsec/lpmetrics.go:389 +0xe8 main.(*MetricsProvider).Run(0x871ea4200, {0xa916d0, 0x871f232c0}) github.com/crowdsecurity/crowdsec/cmd/crowdsec/lpmetrics.go:411 +0x7c main.startLPMetrics.func1() github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:121 +0x3c created by main.startLPMetrics in goroutine 16 github.com/crowdsecurity/crowdsec/cmd/crowdsec/crowdsec.go:119 +0x105

What did you expect to happen?

CrowdSec should not panic/crash - the metrics goroutine should handle a nil value gracefully instead of crashing the whole service.

How can we reproduce it (as minimally and precisely as possible)?

Not reliably reproducible on demand. Observed correlation: an unrelated pfSense process (check_reload_status) was pinned at 100% CPU for ~10 minutes immediately before this crash occurred, possibly following a WAN interface flap. Suggests the panic may be related to timing/resource contention during a reload cycle rather than a fixed input.

Anything else we need to know?

Setup: pfSense CE 2.8.1 (FreeBSD 15) running CrowdSec as one Log Processor, with a second remote Log Processor (separate Linux box) also reporting to this LAPI. LAPI listen address is non-default (bound to LAN IP, not 127.0.0.1). Would appreciate a nil-check guard around whatever datasource/console-option lookup happens in sendMetrics at lpmetrics.go:389, and confirmation whether this is already fixed in a later version.

Crowdsec version

bash
$ cscli version
version: v1.7.8-6322745
Codename: alphaga
BuildDate: 2026-06-17_18:15:38
GoVersion: 1.26.4
Platform: freebsd
libre2: WebAssembly
User-Agent: crowdsec/v1.7.8-6322745-freebsd
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog, db_mysql, db_postgres, db_sqlite

OS version

bash
$ uname -a
FreeBSD pfSense-Dell.local 15.0-CURRENT FreeBSD 15.0-CURRENT #21 RELENG_2_8_1-n256095-47c932dcc0e9: Thu Aug 28 16:27:48 UTC 2025     [email protected]:/var/jenkins/workspace/pfSense-CE-snapshots-2_8_1-main/obj/amd64/AupY3aTL/var/jenkins/workspace/pfSense-CE-snapshots-2_8_1-main/sources/FreeBSD-src-RELENG_2_8_1/amd64.amd64/sys/pfSense amd64

pfSense CE 2.8.1

Enabled collections and parsers

bash
$ cscli hub list -o raw
firewallservices/pf is tainted by scenarios:firewallservices/pf-scan-multi_ports
crowdsecurity/pfsense is tainted by collections:firewallservices/pf
Loaded: 165 parsers, 12 postoverflows, 784 scenarios, 9 contexts, 5 appsec-configs, 222 appsec-rules, 165 collections
Unmanaged items: 0 local, 3 tainted
name,status,version,description,type
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/http-logs,enabled,1.3,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nginx-logs,enabled,2.0,Parse nginx access and error logs,parsers
crowdsecurity/pfsense-gui-logs,enabled,0.1,Parse pfSense web auth logs,parsers
crowdsecurity/public-dns-allowlist,enabled,0.1,Allow events from public DNS servers,parsers
crowdsecurity/sshd-logs,enabled,3.1,Parse openSSH logs,parsers
crowdsecurity/sshd-success-logs,enabled,0.1,Parse successful ssh logins,parsers
crowdsecurity/syslog-logs,enabled,1.0,,parsers
crowdsecurity/whitelists,enabled,0.3,Whitelist events from private ipv4 addresses,parsers
firewallservices/pf-logs,enabled,0.8,Parse packet filter logs,parsers
crowdsecurity/cdn-whitelist,enabled,0.5,Whitelist CDN providers,postoverflows
crowdsecurity/google-special-crawlers-whitelist,enabled,0.1,"Whitelist events from Google special crawlers (e.g. Google-InspectionTool, GoogleOther)",postoverflows
crowdsecurity/rdns,enabled,0.4,Lookup the DNS associated to the source IP only for overflows,postoverflows
crowdsecurity/seo-bots-whitelist,enabled,0.5,Whitelist good search engine crawlers,postoverflows
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.7,Detect cve-2021-44228 exploitation attempts,scenarios
crowdsecurity/ban-defcon-drop_range,enabled,0.2,Ban a range if more than 5 ips from it are banned at a time,scenarios
crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios
crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios
crowdsecurity/CVE-2022-26134,enabled,0.4,Confluence - RCE (CVE-2022-26134),scenarios
crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.4,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios
crowdsecurity/CVE-2023-22518,enabled,0.3,Detect CVE-2023-22518 exploits,scenarios
crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios
crowdsecurity/CVE-2024-0012,enabled,0.1,Detect CVE-2024-0012 exploitation attempts,scenarios
crowdsecurity/CVE-2024-38475,enabled,0.1,Detect CVE-2024-38475 exploitation attempts,scenarios
crowdsecurity/CVE-2024-9474,enabled,0.1,Detect CVE-2024-9474 exploitation attempts,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.3,F5 BIG-IP TMUI - RCE (CVE-2020-5902),scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.4,Detect cve-2018-13379 exploitation attempts,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.3,Grafana - Arbitrary File Read (CVE-2021-43798),scenarios
crowdsecurity/http-admin-interface-probing,enabled,0.5,Detect generic HTTP admin interface probing,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.6,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,1.2,Detect usage of bad User Agent,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.3,Apache - Path Traversal (CVE-2021-41773),scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.3,Apache - Path Traversal (CVE-2021-42013),scenarios
crowdsecurity/http-cve-probing,enabled,0.6,Detect generic HTTP cve probing,scenarios
crowdsecurity/http-generic-bf,enabled,0.9,Detect generic http brute force,scenarios
crowdsecurity/http-generic-test,enabled,0.2,Crowdsec Generic Test Scenario: basic HTTP trigger,scenarios
crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.4,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.4,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sap-interface-probing,enabled,0.1,Detect generic HTTP SAP interface probing,scenarios
crowdsecurity/http-sensitive-files,enabled,0.4,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.4,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-technology-probing,enabled,0.1,Detect HTTP technology/vendor probing,scenarios
crowdsecurity/http-wordpress-scan,enabled,0.4,Detect exploitation attempts against common WordPress endpoints,scenarios
crowdsecurity/http-xss-probing,enabled,0.4,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.4,Detect Atlassian Jira CVE-2021-26086 exploitation attempts,scenarios
crowdsecurity/netgear_rce,enabled,0.4,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios
crowdsecurity/nginx-req-limit-exceeded,enabled,0.3,Detects IPs which violate nginx's user set request limit.,scenarios
crowdsecurity/pfsense-gui-bf,enabled,0.2,Detect bruteforce on pfsense web interface,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.4,Detect cve-2019-11510 exploitation attempts,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios
crowdsecurity/ssh-generic-test,enabled,0.2,Crowdsec Generic Test Scenario: SSH brute force trigger,scenarios
crowdsecurity/ssh-refused-conn,enabled,0.1,Detect sshd refused connections,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios
crowdsecurity/ssh-time-based-bf,enabled,0.3,Detect time-based ssh bruteforce attempts that evade rate limiting (with false positive reduction),scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.7,Detect ThinkPHP CVE-2018-20062 exploitation attempts,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.3,Detect VMSA-2021-0027 exploitation attempts,scenarios
firewallservices/pf-scan-multi_ports,"enabled,tainted",?,Detect aggressive portscans (pf),scenarios
ltsich/http-w00tw00t,enabled,0.3,detect w00tw00t,scenarios
crowdsecurity/bf_base,enabled,0.1,,contexts
crowdsecurity/firewall_base,enabled,0.2,,contexts
crowdsecurity/http_base,enabled,0.3,,contexts
crowdsecurity/base-http-scenarios,enabled,1.4,http common : scanners detection,collections
crowdsecurity/freebsd,enabled,0.5,core freebsd support : syslog+geoip+ssh,collections
crowdsecurity/http-cve,enabled,3.0,Detect CVE exploitation in http logs,collections
crowdsecurity/nginx,enabled,0.3,nginx support : parser and generic http scenarios,collections
crowdsecurity/pfsense,"enabled,tainted",0.3,core pfsense support,collections
crowdsecurity/pfsense-gui,enabled,0.2,pfSense web authentication support,collections
crowdsecurity/sshd,enabled,0.9,sshd support : parser and brute-force detection,collections
crowdsecurity/whitelist-good-actors,enabled,0.4,Good actors whitelists,collections
firewallservices/pf,"enabled,tainted",0.3,Parser and scenario for Packet Filter logs,collections

Acquisition config

bash
# On Linux:
$ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/*
# paste output here

# On Windows:
C:\> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml
# paste output here

Config show

bash
$ cscli config show
# paste output here

Prometheus metrics

bash
$ cscli metrics
# paste output here

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

Source: crowdsecurity/crowdsec

View original on GitHubView discussion on GitHub