[FR] Isolate server-info from exhausted WSGI handler capacity
[!WARNING] Before submitting a PR, please make sure that:
- A maintainer has triaged this issue and applied the
readylabel- This issue has no assignee
- No duplicate PR exists
PRs not meeting these requirements may be automatically closed.
Proposal Summary
Keep the server-info capability-discovery endpoint responsive when the shared WSGI handler thread capacity is exhausted in the default Uvicorn deployment.
Motivation
The default MLflow server runs the FastAPI app under Uvicorn, but GET /api/3.0/mlflow/server-info and its AJAX equivalent are currently Flask handlers behind the catch-all WSGI mount. The mount invokes Flask through anyio.to_thread.run_sync, using the default per-process AnyIO thread-pool limiter shared by Flask-backed API handlers.
When that capacity is fully occupied by slow or blocked Flask requests, server-info also queues. Clients use this endpoint for capability discovery, including workspace support and artifact multipart capabilities, so an otherwise lightweight request can be delayed by unrelated handler load.
Proposed implementation
Register native FastAPI routes for both server-info paths before the Flask mount. Extract the existing payload-building logic from _get_server_info() into a framework-neutral synchronous helper that returns a dictionary:
- Preserve
_get_server_info()and its Flask route for Gunicorn and Waitress deployments; it should wrap the shared helper withjsonify. - Have the FastAPI routes call the helper with
anyio.to_thread.run_sync(..., limiter=<dedicated server-info limiter>)and returnJSONResponse. - Give the dedicated per-process limiter a small reserved capacity (for example, one or two slots), so normal Flask/WSGI requests cannot consume it.
- Preserve the current API and AJAX response shape and error behavior.
This does not aim to guarantee availability during CPU saturation or an event-loop stall. It isolates server-info from exhaustion of the shared WSGI/AnyIO handler capacity without increasing the number of Uvicorn worker processes.
Willingness to contribute
Not specified.
Source: mlflow/mlflow