Race condition in shutdown logic leading to lost published events under certain conditions

Describe the bug The shutdown() method documentation states that it is ensuring that already published events are consumed first before halting the consumers (and of course producers must have been stopped first, else it never returns).

In some circumstances (parallelism = 1 or close to 1, number of items published is less than the ring buffer size), some published events may be trashed (= not consumed by any handler).

This happens whenever none of the threads running the consumer logic reaches the RUNNING state before the shutdown() method scans them (it ignores the non-RUNNING ones).

To Reproduce Reproducer (minimal maven bundle) is provided below, checking if all published events have been consumed and showing in which order things are happening (or not) during the execution using disruptor version 3.4.4 (minor edits necessary to run it with 4.0.0 but reproduces as well).

disruptor-startstop-racecondition.tgz

Extract then run it with:

mvn clean compile exec:java -Dexec.mainClass="issues.Repro"

Sample failing output:

OK => EventHandler.onStart() -> ringBuffer.publish(0) -> EventHandler.onEvent(dummy #0)/begin -> disruptor.shutdown()/start -> EventHandler.onEvent(dummy #0)/done -> disruptor.shutdown()/done -> completionCheck -> EventHandler.onShutdown()
KO => ringBuffer.publish(0) -> disruptor.shutdown()/start -> disruptor.shutdown()/done -> completionCheck
KO => ringBuffer.publish(0) -> disruptor.shutdown()/start -> disruptor.shutdown()/done -> completionCheck
KO => ringBuffer.publish(0) -> disruptor.shutdown()/start -> disruptor.shutdown()/done -> completionCheck
KO => ringBuffer.publish(0) -> disruptor.shutdown()/start -> disruptor.shutdown()/done -> completionCheck

Exit status code is 1.

This repro also contains a workaround for this situation (see below).

Expected behavior All published events must be consumed before shutdown() method returns, even in such a corner case situation.

Desktop (please complete the following information):

• OS: MacOS 26.6.2 (Tahoe)
• Version: 3.4.4 / 4.0.0
• JVM Version: OpenJDK 1.8.0_504, 11.0.26, 17.0.14, 23.0.2

Additional context Note that the first execution is always successful: I suspect that this is because code is running on a cold VM and therefore leaves a lot of room for thread context switching. Once it's warmed up and code is compiled, then this starts to fail (second and subsequent executions).

You will notice that increasing the number of handlers (not included in the repro code) decreases the chance that this happens: the more consumer threads there are, the more chance there is that at least 1 of them reached the RUNNING state before the shutdown() method is called.

Same applies to number of published events - the more events published, the less likely that the problem shows.

Working around the problem The current design allows to workaround the situation once identified, using the LifecycleAware marker interface with 3.x versions, or with the EventHandlerBase interface with 4.x. The threads running the event handler logic will then call the onStart() method after it reached the RUNNING state: it's therefore easy to add a rendez-vous in this method so that the code that started the disruptor waits for all consumer threads to reach this barrier before publishing any event.

This is implemented in class named WorkingAroundHandler in the repro code, and code using it is commented out with the "WORKING AROUND THE PROBLEM" marker.