Simplifying multi-role LLM setups with a single OpenAI-compatible gateway — offering a tested config2.yaml recipe + cost data

Hi MetaGPT team,

First off — the SOP-based multi-agent design (PM → Architect → Engineer → QA with structured handoffs) is still one of the most elegant answers to "how do multi-agent systems avoid degenerating into a chat loop." The per-role LLM configuration in config2.yaml is a killer feature for cost/quality tuning.

The friction point I keep hitting

MetaGPT supports 15+ LLM providers, which is great — but each api_type carries its own config schema. In a single realistic setup you end up with something like:

azure → azure_api_key + azure_endpoint + deployment_id + api_version spark → app_id + api_key + api_secret + websocket base_url qianfan → access_key + secret_key (or api_key + secret_key depending on auth mode) anthropic / gemini / dashscope / zhipuai → each their own field set Now layer on the roles: config where the Architect runs one model, the Engineer another, the QA a third — and a single "software company" run may need 3–4 provider accounts with 3–4 different credential schemas, different base_url path conventions (Ollama wants /api, OpenAI-compatible wants /v1 — a classic source of async for requires an object with aiter errors), and different rate limits. Every team I've helped onboard to MetaGPT spends its first hour debugging YAML, not building agents.

What I've been testing

I maintain AnyLLM, an OpenAI-compatible API gateway that fronts 24+ international and Chinese models (GPT/Claude/Gemini/Qwen/GLM/DeepSeek/Kimi etc.) behind a single API key and a single Base URL, with pass-through pricing.

The interesting part for MetaGPT users is not the product itself — it's the config pattern it enables. Since AnyLLM (like any OpenAI-compatible gateway) speaks one protocol, the entire multi-role setup collapses to one schema: a single api_type: openai block with one base_url and one api_key at the top level, and each entry under roles: reduced to nothing but a model string. The ProductManager can run DeepSeek, the Architect Claude, the Engineer GPT — all authenticated by the same credential, all switched by changing a model name. The same pattern works with LiteLLM, OneAPI, or any OpenAI-compatible proxy — this is a gateway-agnostic recipe.

What I'd like to contribute

A "Single-gateway multi-role setup" recipe for the docs / config2.example.yaml — provider-neutral, showing how to collapse mixed-provider role configs into one OpenAI-compatible endpoint. I've already drafted a working example config and would happily submit it as a PR if maintainers think it fits.

Benchmark + cost data from a real generate_repo run: end-to-end token consumption and per-role cost breakdown across a full SOP pipeline (e.g. "Create a 2048 game"), run through a single gateway with different model mixes — useful context for users tuning max_auto_summarize_code and role/model assignments. I'll share the raw numbers regardless of whether they mention a specific gateway.

Optionally, a note on using GET /v1/models for dynamic model discovery — the config currently requires hardcoding model names per role, and a runtime model list could help tools like MetaGPT WebUI build pickers instead of free-text model fields.

To be transparent: AnyLLM is my product and I benefit from awareness, but the config recipe and benchmark data stand on their own and are vendor-neutral. If there's a preference for documentation-only contributions without product mentions, I'm happy to go that route — tell me what works.

Would a docs PR along these lines be welcome? And is there interest in the multi-role cost benchmark?

Thanks for building the framework that made "agents as a software company" concrete.

Leo Bennett [email protected]