#6596 Concurrent geo-sorted searches on an 11M-document index OOM-kill the process: each search deserializes the entire geo RTree

Concurrent geo-sorted searches on an 11M-document index OOM-kill the process: each search deserializes the entire geo RTree

Summary

On an index of ~11.2M documents each carrying a _geo point, running several geo-sorted searches concurrently makes the meilisearch process allocate multiple GB of anonymous memory per query and get OOM-killed within seconds. A single such query survives (with a ~2–3 GB RSS spike and ~1.8 s latency). The process then restart-loops if the query mix continues.

Root cause appears to be in milli: for any geo-sorted search with ≥1000 candidates (the GeoSortStrategy::Dynamic(1000) default), fill_cache calls index.geo_rtree(txn), which bincode-deserializes the whole-corpus RTree<GeoPoint> from LMDB into anonymous heap memory, per search. The only cache is a field on the per-search GeoSort struct (search/new/geo_sort.rs), so N concurrent geo-sorted searches hold N full copies of the tree.

Environment

meilisearch v1.53.1 and v1.52.3 (official Docker images getmeili/meilisearch:v1.53.1 / :v1.52.3) — identical behavior on both
Single node, container memory limit 4 GiB (mem_limit), host 16 GB RAM with ~7 GB free at incident time
Index: 11,194,620 documents, every doc has _geo; searchable text fields; synonyms configured; all other settings default

Observed behavior

Concurrency-10 benchmark of typeahead-style queries (3/6/10-char prefixes) with "sort": ["_geoPoint(lat,lng):asc"]:

Process RSS climbs ~0.45 GiB → ~4 GiB within seconds, then the kernel kills it:
```
Memory cgroup out of memory: Killed process 1692285 (meilisearch)
total-vm:2175979948kB, anon-rss:4091448kB, file-rss:31200kB
oom-kill:constraint=CONSTRAINT_MEMCG, ... task=meilisearch
```
(10 such kills recorded across two versions and three query variants; total-vm is ~2.17 PB at each kill — presumably dominated by sparse LMDB maps, listed here for completeness.)
Clients see dropped connections (ConnectError / RemoteProtocolError), not HTTP errors.
A single worst-case query (3-char prefix, ~100k+ candidates) in isolation: HTTP 200 in ~1.76 s, RSS spike ~2–3 GiB, process survives.
Bounding the candidate set does not help: adding "filter": "_geoRadius(lat,lng,50000)" still crashes under concurrency, as does restricting to 6/10-char prefixes — consistent with the whole-tree load being independent of the filtered candidate set.
Baselines on the same index without geo sort: p95 61–77 ms at concurrency 10, zero errors, steady RSS ~100 MiB.

Code pointers (current `main`)

crates/milli/src/documents/geo_sort.rs — fill_cache() calls index.geo_rtree(txn)? and inserts the result into self.rtree (per-GeoSort-instance cache only).
crates/milli/src/index.rs — geo_rtree() returns SerdeBincode<RTree<GeoPoint>> deserialized from the main DB: a full owned copy per call.
crates/milli/src/search/new/geo_sort.rs — the GeoSort ranking rule (constructed per search) holds rtree: Option<RTree<GeoPoint>>; no cross-search sharing exists (no Arc/shared cache anywhere in the geo-sort path).

Suspected fix direction

Share a single Arc<RTree<GeoPoint>> per index across concurrent searches (with invalidation on index update), or serve nearest-neighbor iteration from a zero-copy/memory-mapped representation, instead of one owned tree per search. Happy to test a patch against our corpus.

#3581 (_geoPoint sort is slow) — the latency dimension of the same path (>1 s on 6.3M docs; we measure ~1.8 s at 11.2M). This report adds the memory/concurrency dimension: at ≥10 concurrent queries the per-query tree copies exhaust any practical container memory limit and the kernel kills the process.

Reproduction sketch

Create an index with ~11M documents, each with a _geo field (our corpus is Canadian civic addresses; any uniform-ish point cloud should behave the same). Settings: _geo in filterableAttributes and sortableAttributes.
From 10 concurrent connections, issue searches with short q prefixes (2–4 chars, e.g. street-number prefixes that match 10k–1M docs) and "sort": ["_geoPoint(<lat>,<lng>):asc"], limit: 8.
Watch RSS: it reaches a 4 GiB container limit in seconds; journalctl -k shows the memcg OOM kills.

I can share the exact benchmark harness (Python/httpx) if useful.