[Logstash - Investigation] Concurrent `PipelineAction::Create` at boot can raise `MetricStore` `KeyError: key not found`
Author: alexcamsCreated Sep 7, 2026Updated Sep 16, 2026
When several pipelines start concurrently, one pipeline's Create intermittently fails with KeyError: key not found while registering flow metrics. Evidence points to a race condition in metric registration, needs investigation to confirm and to implement the proper fix.
Evidence
- Backtrace (key frames):
KeyError: key not found concurrent/map.rb:in `raise_fetch_no_key' / `fetch' / `fetch_or_store' logstash/instrument/metric_store.rb:in `block in fetch_or_store' logstash/instrument/collector.rb:in `register?' org/logstash/execution/AbstractPipelineExt.java:in `initialize_flow_metrics' logstash/agent.rb:in `block in converge_state' - Intermittent (a different pipeline each time). Signature of a timing race, not a config/compile error.
- At boot each
Createruns on its pwm, so many threads hit the single process-wide collector at once.
Code context
The random, intermittent pipeline error looks like a race condition, but the raised exception comes from a call protected by @structured_lookup_mutex, so I think every structured-store write is serialized. A call serialized that way would behave single-threaded and normally never raise. Resolving that contradiction is the goal of this issue.
- The
fetch_or_storecall in Logstash that raises — metric_store.rb#L85 - Where it throws in
concurrent-ruby(~> 1.3) —raise_fetch_no_keymap.rb#L327
Source: elastic/logstash