#1191·PowerJob

[Bug] MuTransporter.ask creates an unbounded number of scheduler threads

Author: LabelZhouCreated Aug 2, 2026Updated Aug 2, 2026

Describe the bug

When the MU protocol is enabled, every call to MuTransporter.ask creates a new single-threaded scheduled executor to handle the request timeout.

The executor reference is discarded immediately and the executor is never shut down. Its core thread therefore remains alive after the request completes. Continuous MU ask traffic causes the number of pool-N-thread-1 threads and native thread memory usage to grow until the application can reach an OOM.

The problematic code is in:

powerjob-remote/powerjob-remote-impl-mu/src/main/java/tech/powerjob/remote/mu/MuTransporter.java

java
Executors.newSingleThreadScheduledExecutor().schedule(...);

To Reproduce

  1. Start a PowerJob 5.1.2 worker or server with the MU protocol enabled.
  2. Continuously send requests through MuTransporter.ask.
  3. Monitor JVM threads with jcmd <pid> Thread.print or jstack <pid>.
  4. Observe that each request creates a new thread named pool-N-thread-1.
  5. After the timeout task executes or is no longer needed, the executor thread remains waiting in ScheduledThreadPoolExecutor$DelayedWorkQueue.

In one production observation, the process had 4,402 live threads, including 3,631 default pool-* threads and 3,583 unique pool IDs. Most of these threads were waiting in empty ScheduledThreadPoolExecutor$DelayedWorkQueue instances. The live thread count later increased to 4,725 without restarting the process.

Expected behavior

MU requests should use a bounded scheduler owned by the MuTransporter instance instead of creating one executor per request. A completed request should cancel its timeout task, and cancelled tasks should be removed from the scheduler queue.

Applications that do not use the MU protocol should not create the MU timeout scheduler or its thread.

Environment

  • PowerJob Version: 5.1.2
  • Java Version: OpenJDK 11 (the affected module also targets Java 8)
  • OS: Linux

Screenshots

Not applicable.

Additional context

The same per-request executor creation is present on the current master branch at commit 332179de (v5.1.2). A single instance-level ScheduledThreadPoolExecutor with setRemoveOnCancelPolicy(true) is sufficient to preserve the existing asynchronous timeout behavior on Java 8 while preventing unbounded thread growth.