rust-genai

genai

A Native-Protocol Multi-AI Provider Library for Rust

# Current release
genai = "0.6"

# Soon to be released (first half of Sept 2026)
genai = "0.7.0-beta"

See v0.7.0-beta

v0.7.0-beta | v0.6.x | Key Features | Providers | Examples | Chat Options | Usage | TLS Backends | Contribution

genai provides a single, ergonomic Rust API for native-protocol multi-AI provider access, including Anthropic, OpenAI, Gemini, xAI, Ollama, Groq, and more.

Over 200+ LLM models, 26+ LLM providers out of the box, including Ollama for local execution.

Out-of-the-box providers: openai, openai_resp, anthropic, gemini, omlx, ollama, ollama_cloud, vertex, bedrock_api, bedrock_sigv4, github_copilot, opencode_go, groq, together, fireworks, cohere, nebius, mimo, deepseek, minimax, zai, zai_coding, bigmodel, aliyun, qwen_cloud, baidu, moonshot (moonshot.cn), kimi (moonshot.ai), aihubmix, open_router, atlascloud, xai

Common models like OpenAI, Anthropic, Gemini, can be accessed with their simple name gpt-5.6-luna, claude-..., gemini-... and the right adapter/providers will be selected.

Providers and adapters can also be forced with a namespaced model name via [name_space]::[model_name], for example, qwen_cloud::qwen3.8-max

Custom OpenAI-compatible endpoints can be accessed with the built-in genai_n adapter. Set GENAI_{n}_ENDPOINT and optionally GENAI_{n}_API_KEY, then use the genai_{n}::model_name namespace:

GENAI_1_ENDPOINT=https://my-host.example.com/v1/
GENAI_1_API_KEY=sk-your-api-key

let client = Client::new()?;
let chat_res = client.exec_chat("genai_1::my-model", chat_req, None).await?;

For an OMLX deployment, set OMLX_ENDPOINT to the OpenAI-compatible base URL before creating the client. OMLX uses OMLX_API_KEY when authentication is required:

OMLX_ENDPOINT=http://my-omlx-host:8000/v1/
OMLX_API_KEY=your-api-key

Then target the model with the omlx:: namespace:

let client = Client::new()?;
let chat_res = client.exec_chat("omlx::my-model", chat_req, None).await?;

Also supports custom endpoints and auth with ServiceTargetResolver (see examples/c06-target-resolver.rs) to support any other providers.

// Can talk to any models / providers
let client = Client::new()?;

let question = "Why is the sky red?";

let chat_req = ChatRequest::new(vec![
	ChatMessage::system("Answer in one sentence"),
	ChatMessage::user(question),
]);

// Model names can even have a reasoning effort suffix, such as "-high", which will be set, and then removed from name when sent to the provider.
let chat_res = client.exec_chat("gpt-5.4-mini-high", chat_req, None).await?;
	
println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));
	

Docs for LLMs | CHANGELOG | BIG THANKS

v0.7.0-beta

v0.7.0 release target: first half of Sept 2026. See latest v0.7.0-beta releases.

Big release: Many fixes, new capabilities, new providers, and minimal API changes.

Key Breaking Changes:

• Fallible client initialization: Client::new()? and ClientBuilder::build()? return Result (no more Client::default()).
• ReasoningEffort::Zero replaces ReasoningEffort::None ("zero" keyword, "none" backward-compatible alias).

New Providers & Adapters:

• Gemini Interactions API (gemini_ix::): stateful conversation sessions (previous_response_id, store).
• Atlas Cloud (atlascloud::), Qwen Cloud (qwen_cloud::), Kimi (kimi::, kimi-*), OMLX (omlx::, omlx-*).
• Custom adapter (genai_{n}::): generic OpenAI-compatible routing via GENAI_{n}_ENDPOINT and GENAI_{n}_API_KEY.

Key Capabilities & Improvements:

• Declarative provider config via ClientBuilder::append_provider_config and ClientConfig::append_provider_config.
• Stream frame inspection via ChatFrameSink and ChatOptions::with_raw_frame_sink.
• Thought signature and signed thinking block preservation across tool turns (Anthropic, Gemini, OpenRouter).
• Anthropic prompt caching on tools (Tool::with_cache_control) and SSE heartbeat events (ChatStreamEvent::Heartbeat).
• OpenTelemetry GenAI semantic conventions instrumentation behind the otel feature flag.

See all in CHANGELOG and v0.7.0 migration guide for more details.

v0.6.x Released

New provider since v0.6.0: minimax

v0.6.0 release date: 2026-05-23

Here’s what’s new:

• New Adapters:
  • AWS Bedrock (bedrock_api and bedrock_sigv4 adapters)
  • open_router
  • vertex (with Gemini and Anthropic support)
  • github_copilot (GitHub Models API)
  • opencode_go
  • baidu
  • aliyun
  • qwen_cloud (managed Qwen Cloud service at qwencloud.com, using the internal Aliyun-compatible endpoint)
  • moonshot
  • aihubmix
  • ollama_cloud (Ollama Cloud)
• Reasoning effort additions: Added ReasoningEffort::Max for Anthropic and ReasoningEffort::XHigh for OpenAI.
• ProviderConfig for model listing: Client::all_model_names(adapter_kind, provider_config) now accepts endpoint and auth overrides, including remote Ollama hosts and custom OpenAI-compatible model listing.
• Ollama and Ollama Cloud: Now use the native Ollama API protocol.
• Gemini schema compatibility: Gemini and Vertex Gemini structured output and tool schemas now normalize common JSON Schema shapes, including const, nullable schema patterns, additionalProperties, and JSON Schema-only keywords rejected by Vertex.
• Bound adapter clients: ClientBuilder::with_adapter_kind(...) and ClientConfig::with_adapter_kind(...) bind a client to a single provider adapter, which is useful for proxies, gateways, Azure-style deployment names, and OpenAI-compatible providers with nonstandard model names.
• ModelSpec and ServiceTarget: Model arguments can be represented as a model name, explicit ModelIden, or complete ServiceTarget, enabling custom endpoints, auth, and model identity without relying on model-name inference.
• OpenAI Responses stateful sessions: OpenAI Responses supports session continuity with previous_response_id, request store, and returned response_id.
• Chat extra body: ChatOptions::with_extra_body(...) provides a low-level request body extension point for provider-specific fields in OpenAI-compatible chat payloads.
• Tool choice: ChatOptions::with_tool_choice(...) adds provider-neutral tool selection hints for automatic, disabled, required, or specific tool calls.
• Built-in tools and WebSearch: Added typed built-in tool support, including ToolName, ToolConfig, WebSearch, and provider mappings for Anthropic, OpenAI, and Gemini.
• Prompt cache controls: Chat-level CacheControl support adds provider-specific prompt caching options, including OpenAI prompt_cache_key and cache retention. On Anthropic, Tool::with_cache_control marks individual tools, and request-level ChatOptions::with_cache_control auto-applies a breakpoint to the static (tools+system) prefix.
• Updated API: Refined ReasoningContent and StopReason handling (v0.6.0-beta.20), including ContentPart::ReasoningContent and provider stop reasons.
• Perf Improvements: HTTP requests use performance optimizations such as gzip, TCP_NODELAY, and HTTP/2 tuning.
• Numerous fixes, optimizations, and API enhancements.

See v0.5.x to v0.6.x migration

See CHANGELOG

See BIG-THANKS for contributors

Key Features

• Multi-AI provider/model access optimized per provider: native protocols when available, OpenAI-compatible APIs when appropriate or required, and one common Rust API for OpenAI, OpenAI Responses, Anthropic, Gemini, Ollama, Ollama Cloud, OpenCode Go, Groq, xAI, DeepSeek, Cohere, Together, Fireworks, Nebius, Mimo, Zai, BigModel, Aliyun, Atlas Cloud, Google Vertex, and GitHub Copilot (direct chat and streaming) (see examples/c00-readme.rs)
• Image analysis (for OpenAI, Gemini Flash-2, Anthropic) (see examples/c07-image.rs)
• Custom auth/API key (see examples/c02-auth.rs)
• Model aliases (see examples/c05-model-names.rs)
• Custom endpoint, auth, and model identifier (see examples/c06-target-resolver.rs)
• And much more

Examples | Thanks | Library Focus | Changelog | Provider Mapping: ChatOptions | Usage

Model to Adapter Resolution

By default, the library resolves the AdapterKind (AI provider) based on the model name prefix:

• OpenAI: gpt-* (most), o1-*, o3-*, o4-*, chatgpt-*, codex-*
• OpenAI Responses: gpt-5-*, gpt-6-*, gpt-* (containing codex or pro)
• Anthropic: claude-*
• Gemini: gemini-*
• xAI: grok-*
• DeepSeek: deepseek-*
• Moonshot: moonshot-* (moonshot.ai)
• Kimi: kimi* (moonshot.ai)
• Zai: glm-*
• Cohere: command-*, embed-*
• Mimo: mimo-*
• OpenCode Go: Namespace opencode_go:: only
• Atlas Cloud: Namespace atlascloud:: only
• Qwen Cloud: Namespace qwen_cloud:: only, for the managed Qwen Cloud service at qwencloud.com. It uses the internal Aliyun-compatible endpoint and QWEN_CLOUD_API_KEY.
• Fireworks: Models containing fireworks
• Ollama: Fallback for any other names, defaulting to local Ollama.

Namespacing (Forcing an Adapter)

You can force a specific adapter by using the adapter_kind::model_name syntax. This is the recommended way for many providers and for disambiguating OpenAI-compatible services.

• groq::openai/gpt-oss-20b (Forces Groq adapter)
• together::meta-llama/Llama-3-8b-chat-hf (Forces Together adapter)
• fireworks::glm-5p1 (for fireworks.ai)
• ollama_cloud::gemma3:4b (Forces Ollama Cloud adapter)
• github_copilot::openai/gpt-5.4-mini (Forces GitHub Copilot adapter)
• nebius::Qwen/Qwen3-235B-A22B (Forces Nebius adapter)
• aliyun::qwen-plus (Forces Aliyun adapter)
• qwen_cloud::qwen-plus (Forces Qwen Cloud adapter, managed by qwencloud.com and backed by the internal Aliyun-compatible service)
• vertex::gemini-2.5-flash (Forces Google Vertex adapter)
• moonshot::moonshot-v1-8k (Forces Moonshot adapter)
• baidu::ernie-4.0 (Forces Baidu adapter)
• zai_coding::glm-4.6 (Special namespace for Zai coding subscription)
• opencode_go::minimax-m2.5 (Forces OpenCode Go adapter)
• bedrock_api::anthropic.claude-v2 (Forces AWS Bedrock adapter)
• open_router::google/gemini-2.0-flash-001 (Forces OpenRouter adapter)
• atlascloud::qwen/qwen3.5-flash (Forces Atlas Cloud adapter)
• genai_1::my-model-7b (Forces Custom adapter with index 1)

For a complete list of AdapterKind, see the AdapterKind enum.

Examples

examples/c00-readme.rs

…

More Examples

• examples/c00-readme.rs - Quick overview code with multiple providers and streaming.
• examples/c01-conv.rs - Shows how to build a conversation flow.
• examples/c02-auth.rs - Demonstrates how to provide a custom AuthResolver to supply auth data, such as api_key, per adapter kind.
• examples/c03-mapper.rs - Demonstrates how to provide a custom AdapterKindResolver to customize the "model name