通过将 LLM 路由器设置为 Cursor 的基础 URL,来访问来自 OpenAI、Groq、本地 Ollama 等模型
Access models from OpenAI, Groq, local Ollama, and other providers by setting LLM-router as the base URL in Cursor.
LLM-router is a reverse proxy that routes chat/completions API requests to various OpenAI-compatible backends based on the model's prefix.
Cursor lacks support for local models and the flexibility to switch between multiple LLM providers efficiently. There's clear demand in the community, evidenced by months of unanswered requests for such features on Cursor's GitHub and Discord channels.
The specific challenges with Cursor include:
Override OpenAI Base URL setting requires a URL that is publicly accessible and secured with https, complicating the use of local models.LLM-router overcomes these limitations, allowing seamless switching between locally served models like Ollama and external services such as OpenAI and Groq.
https://github.com/kcolemangt/llm-router/assets/20099734/7220a3ac-11c5-4c89-984a-29d1ea850d10
Create your configuration file:
config-sample.json is provided in the repositoryconfig.json or create a new config.json file using the example belowLaunch LLM-router to manage API requests across multiple backends:
./llm-router-darwin-arm64LLMROUTER_API_KEY environment variable, the program will generate a random strong API keyngrok http 11411https://xxxx.ngrok-free.app)Configure Cursor to use your LLM-router:
Define your preferred models in Cursor using the appropriate prefixes:
ollama/phi4
openai/gpt-4o-mini
groq/deepseek-r1-distill-llama-70b-specdec⚠️ Important Warning: When clicking "Verify", Cursor randomly selects one of your enabled models to test the connection. Make sure to uncheck any models in Cursor's model list that aren't provided by the backends configured in your config.json. Otherwise, verification may fail if Cursor tries to test a model that's not available through your LLM-router.
Here is an example of how to configure Groq, Ollama, and OpenAI backends in config.json:
…In this configuration, OpenAI serves as the default backend, allowing you to use model identifiers like openai/gpt-4o-mini or simply gpt-4o-mini. Models on Ollama and Groq, however, must be prefixed with ollama/ and groq/ respectively. This configuration also causes Cursor to send optimized reasoning prompts to Groq's deepseek-r1-distill-llama-70b-specdec model and the local Ollama qwq model.
Clients such as Cursor send specialized prompts to specific models recognized for enhanced reasoning performance. Usually, these optimized prompts target proprietary models. The Model Aliases and Role Rewrites features in LLM-router allow you to extend these optimizations to reasoning-oriented models hosted locally (such as via Ollama) or by alternative providers (like Groq).
By combining aliases and role rewrites, you can route optimized reasoning prompts effectively across different backend models.
Aliases allow mapping specific client-recognized reasoning model identifiers to backend models of your choice. This causes Cursor and similar clients to use specialized reasoning prompts designed for those models.
Simple example:
"aliases": {
"o1": "groq/deepseek-r1-distill-llama-70b-specdec"
}In this example:
o1, but LLM-router redirects these requests to Groq's reasoning-focused model deepseek-r1-distill-llama-70b-specdec.gpt-4-turbo.Role rewrites ensure that message roles from clients are correctly translated to match backend provider expectations. Clients using specialized reasoning prompts often use custom roles (e.g., developer) that may not be recognized universally.
Simple example:
"role_rewrites": {
"developer": "system"
}developer from Cursor will be rewritten to the standard role system, ensuring compatibility with backends.Each model provider may support different parameters in their API requests. The unsupported_params option allows you to specify parameters that should be automatically removed from requests before forwarding them to specific backends, preventing errors due to incompatible parameters.
Simple example:
"unsupported_params": [
"reasoning_effort"
]In this example:
reasoning_effort will be automatically removed from requests before forwarding to this backend.Here's a complete configuration example illustrating how these features are used together:
…In this configuration:
o1 from Cursor go to Groq's reasoning-oriented model deepseek-r1-distill-llama-70b-specdec.o3-mini go to the local Ollama model qwq.developer role to the standard system role.reasoning_effort parameter, which is not supported by Groq's API.These features are not limited to reasoning models. They can be applied broadly to facilitate compatibility and optimal prompting strategies across various model types and backend configurations.
Provide the necessary API keys via environment variables:
OPENAI_API_KEY= GROQ_API_KEY= ./llm-router-darwin-arm64If you wish to specify your own LLM-router API key instead of using a generated one:
LLMROUTER_API_KEY=your_custom_key GROQ_API_KEY= ./llm-router-darwin-arm64Alternatively, you can use the command-line flag:
./llm-router-darwin-arm64 --llmrouter-api-key=your_custom_keyYou can also store your API keys and other configuration in a .env file in the same directory as LLM-router. This is recommended to avoid exposing sensitive keys in your shell history or environment.
.env file by copying the provided example:cp .env.example .env.env file with your API keys:# LLM-Router configuration
LLMROUTER_API_KEY=your_llmrouter_api_key_here
OPENAI_API_KEY=your_openai_api_key_here
GROQ_API_KEY=your_groq_api_key_hereLLM-router will automatically load variables from this file at startup. Environment variables that are already set will take precedence over those in the .env file, following standard precedence rules.
Routes chat/completions API requests to any OpenAI-compatible LLM backend based on the model's prefix. Streaming is supported.
LLM-router can be set up to use individual API keys for each backend, or no key for services like local Ollama.
Requests to LLM-router are secured with your LLMROUTER_API_KEY which you set in Cursor's OpenAI API Key field. This key is used to authenticate requests to your LLM-router, while the backend-specific API keys (like GROQ_API_KEY) are used by LLM-router to authenticate with the respective API providers.
When attempting to run LLM-router on MacOS, you may encounter permissions errors due to MacOS's Gatekeeper security feature. Here are several methods to resolve these issues and successfully launch the application.
If you receive a warning about permissions after downloading the release binary, change the file's permissions to make it executable:
chmod +x llm-router-darwin-arm64
./llm-router-darwin-arm64spctl CommandIf the above does not resolve the issue and you still face security pop-ups:
Add the application to the allowed list using:
sudo spctl --add llm-router-darwin-arm64Attempt to run the application again:
./llm-router-darwin-arm64For issues persisting beyond previous steps:
llm-router-darwin-arm64 in Finder.This step should register the application as a trusted entity on your Mac, bypassing Gatekeeper on subsequent launches.
Should the above methods fail:
If none of the above methods work, consider building the application from source:
make
./build/llm-router-localX (twitter) @kcolemangt
LinkedIn Keith
暂无开放 Issues,或尚未同步最近议题。