#19489·logstash

[Logstash - Investigation] Concurrent `PipelineAction::Create` at boot can raise `MetricStore` `KeyError: key not found`

Author: alexcamsCreated Sep 7, 2026Updated Sep 16, 2026

When several pipelines start concurrently, one pipeline's Create intermittently fails with KeyError: key not found while registering flow metrics. Evidence points to a race condition in metric registration, needs investigation to confirm and to implement the proper fix.

Evidence

  • Backtrace (key frames):
    KeyError: key not found
      concurrent/map.rb:in `raise_fetch_no_key' / `fetch' / `fetch_or_store'
      logstash/instrument/metric_store.rb:in `block in fetch_or_store'
      logstash/instrument/collector.rb:in `register?'
      org/logstash/execution/AbstractPipelineExt.java:in `initialize_flow_metrics'
      logstash/agent.rb:in `block in converge_state'
  • Intermittent (a different pipeline each time). Signature of a timing race, not a config/compile error.
  • At boot each Create runs on its pwm, so many threads hit the single process-wide collector at once.

Code context

The random, intermittent pipeline error looks like a race condition, but the raised exception comes from a call protected by @structured_lookup_mutex, so I think every structured-store write is serialized. A call serialized that way would behave single-threaded and normally never raise. Resolving that contradiction is the goal of this issue.