Real memory breaker's G1 GC trigger fills free regions with humongous arrays, causing evacuation failures
Description
When the real memory parent breaker is about to trip, G1OverLimitStrategy (#58674) "gently nudges G1 to do a young
GC". It allocates half-region arrays, one humongous region each, up to one per free region plus one, until it observes
a GC. Each humongous allocation takes a free region directly, including regions G1 needs as to-space. The young GC that
the burst eventually forces can then fail evacuation.
Why it matters
The point of the trigger is a young GC that reclaims memory. When that GC fails evacuation, the chance is lost: the
node stays over the limit, and the breaker falls back to System.gc() (#102563). With ES defaults, JvmErgonomics
sets G1ReservePercent=25 for heaps of 8 GiB and up (#59667), while the breaker trips at 95%. So near the limit, the
free heap is already smaller than the reserve.
Evidence
My own GC logs from production nodes, taken before any change: 120 GiB heap, 32 MiB regions,
indices.breaker.total.limit: 85%, G1ReservePercent=10. Each trigger burst allocated 490–570 humongous regions. It
was followed by To-space exhausted / evacuation failure, and then Pause Full (System.gc()).
Proposal
Keep one humongous allocation, and make every later allocation a quarter-region eden array under a budget derived from the same free-region estimate. The loop still stops at the first GC it observes. I'll open a PR with the change and results from a synthetic reproduction.
Opened as #160450.
Source: elastic/elasticsearch