[FR] Add OpenAI-compatible `GET /v1/models` for MLflow Gateway endpoints
[!WARNING] Before submitting a PR, please make sure that:
- A maintainer has triaged this issue and applied the
readylabel- This issue has no assignee
- No duplicate PR exists
PRs not meeting these requirements may be automatically closed.
Willingness to contribute
Yes. I can contribute this feature independently.
Proposal Summary
MLflow Gateway already exposes OpenAI-compatible inference routes such as POST /v1/chat/completions, but it does not expose the corresponding OpenAI-style model discovery endpoint, GET /v1/models.
This creates a compatibility gap for OpenAI-compatible clients and tools that expect both inference APIs and a model-listing API.
Motivation
In some OpenAI-compatible integrations, support for chat or responses APIs alone is not sufficient. Some clients, SDK wrappers, agent frameworks, gateways, and developer tools also call GET /v1/models to discover available models, validate configuration, or populate model selectors.
Today, an MLflow Gateway endpoint can be invoked via the OpenAI-compatible API, but clients that also require /v1/models may fail to initialize or require MLflow-specific workarounds.
Details
Add an OpenAI-compatible GET /v1/models endpoint for MLflow Gateway so that newly created endpoints can be discovered in a standard way, not only invoked.
This would make MLflow Gateway more interoperable with the broader OpenAI-compatible ecosystem.
If this is a feature the maintainers would be open to supporting, I’d be happy to contribute a PR.
What machine learning domain(s) is this feature request about?
-
domain/genai: LLMs, Agents, and other GenAI-related use cases -
domain/classical-ml: Traditional machine learning, such as linear regression. -
domain/deep-learning: Deep learning and neural networks. -
domain/platform: MLflow platform foundation, not specific to a particular machine learning domain.
What area(s) of MLflow is this feature request about?
-
area/tracking: Tracking Service, tracking client APIs, autologging -
area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry -
area/scoring: MLflow model serving, deployment tools, Spark UDFs -
area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows -
area/prompt: MLflow prompt engineering features, prompt templates, and prompt management -
area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality -
area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations -
area/projects: MLproject format, project running backends -
area/uiux: Front-end, user experience, plotting -
area/docs: MLflow documentation pages
Source: mlflow/mlflow