Gozar

Gozar

Gozar is a self-hosted, OpenAI-compatible LLM gateway for local projects, private teams, and developer workflows. Applications use one stable /v1 endpoint and a Gozar API key while the gateway routes requests through operator-managed upstream accounts, provider API keys, and fallback chains.

Use Gozar when you want a private, Docker-first, OpenAI-compatible proxy that can be used from the OpenAI SDK, LangChain, LangGraph, Postman, cURL, internal tools, local agents, and project-specific backends without rewriting every client integration.

Current release: 0.1.0. Gozar is source-available under the PolyForm Noncommercial License 1.0.0 and is intended for self-hosted, non-commercial use.

Why Gozar?

Developers often need a local or self-hosted LLM gateway that behaves like the OpenAI API but still lets them control routing, credentials, limits, traces, and fallbacks. Gozar gives every project a single /v1 endpoint and a per-project Gozar API key, while the operator manages upstream credentials in one place.

What You Get

• Drop-in OpenAI compatibility - use /v1/chat/completions, /v1/embeddings, streaming SSE, and /v1/models with standard OpenAI-style shapes.
• One API key per app or workflow - issue Gozar API keys for each local project, agent, backend service, or team integration.
• Bring your own upstream access - connect API-key providers such as OpenAI and OpenRouter, plus subscription providers such as Codex and Anthropic where supported by the deployment.
• Codex device-code sign-in - Codex subscription connect does not depend on a broken localhost redirect. Gozar shows a one-time code and completes the account connection after OpenAI approval.
• Two-lane fallback chains - one chain ID contains an LLM lane and an Embeddings lane. Every node selects its own account, model, and fallback policy.
• Chain health alerts - saved routes are rechecked against current account status and model catalogs; removed models and unavailable accounts are surfaced before they become silent production failures.
• Route-aware model discovery - Chat and Embeddings catalogs are discovered, cached, and refreshed independently for each API-key account; subscription providers can use runtime fallback catalogs.
• LangChain and LangGraph friendly - point ChatOpenAI at the Gozar /v1 base URL and use the Gozar API key. Chain selection happens inside Gozar.
• Usage limits, traces, and analytics - track request volume, token usage, per-token activity, per-account activity, and routing outcomes.
• Secure by default - encrypted credential storage, fail-closed operator auth, secret-free logs, password-confirmed API key reveal, Docker-first deployment, and production reverse-proxy guidance.

Contents

• How Gozar Works
• Quick Start: Run Locally with Docker
• First-Run Setup in the Console
• Connect Upstream Accounts
• Build a Provider-Aware Fallback Chain
• Create a Gozar API Key
• Dynamic Chains and Per-Call Overrides
• Use Gozar from Your App
• Model Discovery
• Admin API
• Architecture
• Production Deployment
• Security
• Configuration
• Troubleshooting
• Development
• Contributing and Security
• Operator Responsibilities
• Disclaimer
• License

How Gozar Works

Gozar has two surfaces:

• Data path: /v1 - the OpenAI-compatible endpoint used by your applications. It accepts a Gozar API key, chooses the correct fallback chain, calls an upstream provider account, and returns an OpenAI-shaped response.
• Control path: /api - the authenticated admin API used by the web console for accounts, API keys, fallback chains, model catalogs, traces, and analytics.

…

The client does not need to know which upstream account was used. It only needs:

GOZAR_BASE_URL=https://your-gozar-domain.example/v1
GOZAR_API_KEY=gz-...
GOZAR_MODEL=

Quick Start: Run Locally with Docker

This is the fastest way to run Gozar for a local project, internal tool, or private development environment.

1. Clone and configure

Clone or download this repository, open its root directory, then run:

cp .env.example .env

Generate strong secrets and place them in .env:

python3 - 
- Backend API: 
- API docs: 
- OpenAPI JSON: 

### 3. Check readiness

```bash
curl http://localhost:8000/health
curl http://localhost:8000/ready

Expected ready response:

{"status":"ready"}

…

bash
curl -X PUT "$GOZAR_ADMIN_BASE_URL/api/chains/by-key/support-production" \
  -H "Authorization: Bearer $GOZAR_ADMIN_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Support production",
    "entries": [
      {
        "account_id": "OPENAI_ACCOUNT_UUID",
        "model": "PRIMARY_CHAT_MODEL_ID",
        "fallback_policy": "auth_or_retryable",
        "route": "chat"
      },
      {
        "account_id": "OPENROUTER_ACCOUNT_UUID",
        "model": "OPENROUTER_CHAT_MODEL_ID",
        "route": "chat"
      },
      {
        "account_id": "OPENROUTER_ACCOUNT_UUID",
        "model": "OPENROUTER_EMBEDDING_MODEL_ID",
        "route": "embeddings"
      }
    ]
  }'

Use the returned chain_id for one request. Routing precedence is:

1. Per-call chain override.
2. Chain pinned to the Gozar API key.
3. Legacy exact model selector.
4. Legacy catch-all chain.

The override can be sent as X-Gozar-Chain-ID or as {"gozar":{"chain_id":"..."}} in SDK extra_body. Gozar removes the private gozar field before calling the upstream provider.

Use Gozar from Your App

The Gozar base URL must include /v1.

For local Docker:

export GOZAR_BASE_URL="http://localhost:8000/v1"
export GOZAR_API_KEY="gz-YOUR_GOZAR_API_KEY"
export GOZAR_MODEL="MODEL_RETURNED_BY_V1_MODELS"
export GOZAR_EMBEDDING_MODEL="PROVIDER_EMBEDDING_MODEL"
export GOZAR_CHAIN_ID="OPTIONAL_CHAIN_UUID"

For production:

export GOZAR_BASE_URL="https://gozar.example.com/v1"
export GOZAR_API_KEY="gz-YOUR_GOZAR_API_KEY"
export GOZAR_MODEL="MODEL_RETURNED_BY_V1_MODELS"
export GOZAR_EMBEDDING_MODEL="PROVIDER_EMBEDDING_MODEL"
export GOZAR_CHAIN_ID="OPTIONAL_CHAIN_UUID"

cURL

curl "$GOZAR_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $GOZAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$GOZAR_MODEL\",
    \"messages\": [
      {\"role\": \"user\", \"content\": \"Hello from Gozar\"}
    ]
  }"

Streaming

curl "$GOZAR_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $GOZAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$GOZAR_MODEL\",
    \"messages\": [
      {\"role\": \"user\", \"content\": \"Stream a short answer\"}
    ],
    \"stream\": true
  }"

Embeddings for RAG and vector memory

POST /v1/embeddings follows the standard OpenAI request and response contract. It uses the same Gozar API key, assigned chain, per-call chain override, limits, usage records, and traces as Chat Completions, but automatically selects the chain's Embeddings lane.

curl "$GOZAR_BASE_URL/embeddings" \
  -H "Authorization: Bearer $GOZAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$GOZAR_EMBEDDING_MODEL\",
    \"input\": [\"first document\", \"second document\"],
    \"encoding_format\": \"float\"
  }"

Embedding nodes accept only embedding-capable OpenAI or OpenRouter API-key accounts. Each node may store a different provider model, for example text-embedding-3-small on OpenAI and openai/text-embedding-3-small on OpenRouter. The node model overrides the inbound model for that attempt, so fallback between providers with different model IDs remains transparent. A blank node model forwards the caller's model unchanged. Gozar never synthesizes a placeholder vector.

response = client.embeddings.create(
    model=os.environ["GOZAR_EMBEDDING_MODEL"],
    input=["first document", "second document"],
    encoding_format="float",
)

vectors = [item.embedding for item in response.data]

See the official OpenAI Embeddings API and OpenRouter Embeddings API for provider model and input details.

OpenAI Python SDK

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["GOZAR_BASE_URL"],
    api_key=os.environ["GOZAR_API_KEY"],
)

response = client.chat.completions.create(
    model=os.environ["GOZAR_MODEL"],
    messages=[{"role": "user", "content": "Hello from Gozar"}],
    extra_headers={"X-Gozar-Chain-ID": os.environ["GOZAR_CHAIN_ID"]},
)

print(response.choices[0].message.content)

OpenAI JavaScript SDK

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.GOZAR_BASE_URL,
  apiKey: process.env.GOZAR_API_KEY,
});

const response = await client.chat.completions.create({
  model: process.env.GOZAR_MODEL ?? "MODEL_RETURNED_BY_V1_MODELS",
  messages: [{ role: "user", content: "Hello from Gozar" }],
});

console.log(response.choices[0]?.message?.content);

LangChain and LangGraph

Use the same /v1 base URL with ChatOpenAI. Your LangGraph node does not need Gozar-specific routing code; the Gozar API key controls chain selection.

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model=os.environ["GOZAR_MODEL"],
    base_url=os.environ["GOZAR_BASE_URL"],
    api_key=os.environ["GOZAR_API_KEY"],
    default_headers={"X-Gozar-Chain-ID": os.environ["GOZAR_CHAIN_ID"]},
    use_responses_api=False,
)

def llm_node(state):
    return {"messages": [llm.invoke(state["messages"])]}

…

json
{
  "model": "MODEL_RETURNED_BY_V1_MODELS",
  "messages": [{"role": "user", "content": "Hello"}],
  "gozar": {"include_metadata": true}
}

…

bash
curl "$GOZAR_BASE_URL/models" \
  -H "Authorization: Bearer $GOZAR_API_KEY"

…

bash
curl "$GOZAR_ADMIN_BASE_URL/api/models/providers/codex" \
  -X PUT \
  -H "Authorization: Bearer $GOZAR_ADMIN_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"models":["gpt-5.5","gpt-5.4-mini"]}'

Reset a provider to the environment default:

curl "$GOZAR_ADMIN_BASE_URL/api/models/providers/codex" \
  -X DELETE \
  -H "Authorization: Bearer $GOZAR_ADMIN_SESSION_TOKEN"

Admin API

The admin API lives under /api and requires an operator session, except for login and first-run bootstrap.

Authenticate:

curl "$GOZAR_ADMIN_BASE_URL/api/auth/login" \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"YOUR_PASSWORD"}'

Use the returned access token:

Authorization: Bearer 

…

mermaid
flowchart TD
    Console[React Web Console] --> API[/Admin API /api/]
    SDK[OpenAI SDK, LangGraph, cURL] --> V1[/OpenAI-compatible API /v1/]
    API --> DB[(PostgreSQL)]
    API --> Redis[(Redis)]
    V1 --> Gate[API key validation and limits]
    Gate --> Router[Fallback router]
    Router --> Creds[Encrypted upstream credentials]
    Creds --> Providers[OpenAI, OpenRouter, Codex, Anthropic]
    V1 --> Trace[Trace and usage recording]
    Trace --> DB
    Trace --> Redis

…

bash
cp .env.example .env
docker compose -f compose.prod.yml up -d --build

In production compose:

• backend binds to 127.0.0.1:8000
• frontend binds to 127.0.0.1:8080
• PostgreSQL and Redis are inter