Flaky test Triage - 2026-09-16

This issue was generated automatically by Claude Code (Anthropic's AI coding agent) running a scheduled CI-triage routine on behalf of @FrankChen021. Analysis and suggested fixes are AI-produced; please verify before acting on them.

This triage covers the 8 commits merged to master on 2026-09-15. Two commits had at least one failed job (three failed jobs in total), plus the scheduled Cron Job ITs run on one of them. None of the failures is caused by the commit itself: one is a known flaky EasyMock race that has already been triaged in #20312 and passed on the neighbouring master commits before and after, one is a transient TLS error while downloading the JFR profiler agent that #20318 (merged 23 seconds later) already fixes, and one is the OWASP dependency-check job flagging memcached server CVEs against the Java memcached client jar, a false-positive CPE match that has failed every recent cron run and is addressed by #20236. No re-runs had been triggered on any of the failed jobs. The Unit & Integration tests run for 4dc0437e0f (#20311) shows as cancelled for three shards; it was superseded by the next master push and is not a failure.

Summary

Commit	Failed job	Failure log	Root cause	Verdict
76ddcf297e (#20307, perf(test): reduce compaction / re-sharding test time)	unit tests (25, S*)	job 104262934243	SeekableStreamSupervisorStateTest.testSupervisorStopTaskGroupEarly: EasyMock verifyAll() reports TaskQueue.shutdown("id1", ...) expected 1, actual 0; failed the initial run and all 3 surefire retries (4/4)	Flaky (race)
76ddcf297e (#20307, perf(test): reduce compaction / re-sharding test time)	openrewrite	job 104262932923	Prepare GITHUB_ENV step: setup_test_profiling_env.sh exited with curl exit code 35 (TLS connect error) while downloading the JFR profiler agent from static.imply.io; the job aborted before Maven started	Infra (fix in #20318)
4395ef3464 (#20330, build(deps): bump aws.sdk.v2.version to 2.54.14)	security vulnerabilities (cron)	job 104239165174	dependency-check-maven:13.0.0:check on druid-server: elasticache-java-cluster-client-1.2.4.jar matched to cpe:2.3:a:memcached:memcached:1.2.4 and flagged with 13 memcached server CVEs (CVE-2016-8704 9.8, CVE-2016-8705 9.8, CVE-2026-47783 8.1, CVE-2026-47784 8.1, ...)	Persistent (fix in #20236)

Analysis and suggested fixes

1. SeekableStreamSupervisorStateTest.testSupervisorStopTaskGroupEarly (indexing-service, S* shard)

The test calls supervisor.handoffTaskGroupsEarly(List.of(0)), which only enqueues a HandoffTaskGroupsNotice on the supervisor's notice queue. It then waits with while (supervisor.getNoticesQueueSize() > 0) Thread.sleep(100) before calling runInternal() and verifyAll(). getNoticesQueueSize() returns notices.size(), and the notice-processing thread does notices.poll(...) and only afterwards calls notice.handle(). So the queue size drops to 0 the moment the notice is taken, not when taskGroup.setHandoffEarly() has actually run. Under load the test thread can win the race, runInternal() sees a task group that is not yet marked for early handoff, and TaskQueue.shutdown("id1", ...) is never invoked, producing exactly the expected: 1, actual: 0 verify failure. Because the race is deterministic for a given scheduling, all four surefire attempts failed in the same way.

This is the same failure already triaged as item 9 in #20312 (hit 1f68c5e168 on 2026-09-10 and PR #20277). Commit 76ddcf297e only touches compaction task tests (CompactionTaskRunBase, NativeCompactionTaskRunTest, MSQCompactionTaskRunTest), the CI shard balancing script and ci.yml; it does not touch the seekable-stream supervisor or this test. The same S* shard passed on the master commits immediately before (4dc0437e0f) and after (997a2a9568, 97fcd37ce7). No fix PR is open yet.

Suggested fix (test-only): do not use queue emptiness as a completion signal. emitNoticeProcessTime(noticeType, ...) is protected and is invoked on the notice thread immediately after notice.handle() returns, so subclass TestSeekableStreamSupervisor to count down a CountDownLatch when noticeType is handoff_task_group_notice, and replace the while (getNoticesQueueSize() > 0) sleep(100) loop with latch.await(timeout) plus an assertion that it did not time out. This both proves the handoff notice has completed and provides the happens-before edge for handoffEarly, which is a non-volatile boolean. The neighbouring lag tests in the same file already use this latch-in-subclass pattern. Overriding addNotice or running the notice synchronously is not possible from a test subclass because both addNotice and HandoffTaskGroupsNotice are private, and polling getHandoffEarly() from the test thread has no visibility guarantee.

Edited 2026-09-16: suggested fix revised after review; the original options (a) and (c) were not viable.

2. openrewrite job: Prepare GITHUB_ENV failed with curl exit code 35 (Static Checks CI)

The Prepare GITHUB_ENV step runs .github/scripts/setup_test_profiling_env.sh, which at 76ddcf297e did a bare curl https://static.imply.io/cp/<agent>.jar -s -o ... under set -e. Curl exit code 35 is an SSL/TLS connect error on the CDN side, so the step aborted 25 seconds into the job, before the OpenRewrite check itself ran. The profiler agent is observability only and has no bearing on the OpenRewrite result. The Static Checks runs for the neighbouring master commits (4dc0437e0f twelve seconds earlier, 997a2a9568 eleven seconds later) passed the same step.

Suggested fix: already merged in #20318 (97fcd37ce7), which adds --retry 3 --retry-all-errors with bounded timeouts, downloads to a temporary file, and falls back to running without the agent while emitting a workflow warning instead of failing the job. No further action needed; if the failure recurs on commits after 97fcd37ce7 the retry/fallback logic should be re-checked.

3. security vulnerabilities (cron): false-positive memcached CVEs on elasticache-java-cluster-client-1.2.4.jar

OWASP dependency-check maps com.amazonaws:elasticache-java-cluster-client:1.2.4 to cpe:2.3:a:memcached:memcached:1.2.4, the C memcached server daemon, purely because the version strings coincide. All 13 reported CVEs (including the newly published CVE-2026-47783 and CVE-2026-47784) are server-side memcached vulnerabilities; Druid only uses this jar as a client library in the memcached cache extension and never runs a memcached process. The dependency was bumped from 1.2.0 to 1.2.4 by Dependabot in #20026 (2026-08-14), and the version-specific CPE match started after that. There is no suppression entry for this artifact in owasp-dependency-check-suppressions.xml on master. The same finding also failed the previous day's cron run (bba74f6cb2, 2026-09-14), so this job is red on every recent cron run and will stay red until a suppression lands; it is unrelated to #20330 (an AWS SDK v2 bump), which merely happened to be the head of master when the cron fired.

Suggested fix: merge #20236, which adds a <suppress> entry matching ^pkg:maven/com\.amazonaws/elasticache-java-cluster-client@.*$ against cpe:/a:memcached:memcached with an explanatory note. If #20236 is blocked on other content, cherry-pick just that suppression block into a standalone PR so the cron job goes green again.