Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#160449·elasticsearch

Real memory breaker's G1 GC trigger fills free regions with humongous arrays, causing evacuation failures

Author: uchiha-ashaCreated Sep 28, 2026Updated Sep 28, 2026
Labels:Core/Infra/Circuit BreakersTeam:Core/Infra

Description

When the real memory parent breaker is about to trip, G1OverLimitStrategy (#58674) "gently nudges G1 to do a young GC". It allocates half-region arrays, one humongous region each, up to one per free region plus one, until it observes a GC. Each humongous allocation takes a free region directly, including regions G1 needs as to-space. The young GC that the burst eventually forces can then fail evacuation.

Why it matters

The point of the trigger is a young GC that reclaims memory. When that GC fails evacuation, the chance is lost: the node stays over the limit, and the breaker falls back to System.gc() (#102563). With ES defaults, JvmErgonomics sets G1ReservePercent=25 for heaps of 8 GiB and up (#59667), while the breaker trips at 95%. So near the limit, the free heap is already smaller than the reserve.

Evidence

My own GC logs from production nodes, taken before any change: 120 GiB heap, 32 MiB regions, indices.breaker.total.limit: 85%, G1ReservePercent=10. Each trigger burst allocated 490–570 humongous regions. It was followed by To-space exhausted / evacuation failure, and then Pause Full (System.gc()).

Proposal

Keep one humongous allocation, and make every later allocation a quarter-region eden array under a budget derived from the same free-region estimate. The loop still stops at the first GC it observes. I'll open a PR with the change and results from a synthetic reproduction.

Opened as #160450.

Source: elastic/elasticsearch

View original on GitHubView discussion on GitHub