LLMTornado

✨ Features

• Use Any Provider: Built-in connectors to: Alibaba, Anthropic, Azure, Blablador, Cohere, DeepInfra, DeepSeek, Google, Groq, LiteLLM, MiniMax, Mistral, MoonshotAI, OpenAI, OpenRouter, Perplexity, Requesty, Upstage, Voyage, xAI, Z.ai, and more. Connectors expose all niche/unique features via strongly typed code and are up-to-date with the latest AI development. No dependencies on first-party SDKs. Feature Matrix tracks detailed endpoint support.
• First-class Local Deployments: Run with vLLM, Ollama, or LocalAI with integrated support for request transformations.
• Agents Orchestration: Coordinate specialist agents that can autonomously perform complex tasks with three core concepts: Orchestrator (graph), Runner (node), and Advancer (edge). Comes with handoffs, parallel execution, Mermaid export, and builder pattern to keep it simple.
• Rapid Development: Write pipelines once, execute with any Provider by changing the model's name. Connect your editor to Context7 or FSKB to accelerate coding with instant access to vectorized documentation.
• Fully Multimodal: Text, images, videos, documents, URLs, and audio inputs/outputs are supported.
• Cutting Edge Protocols:
  • MCP: Connect agents to data sources, tools, and workflows via Model Context Protocol with LlmTornado.Mcp.
  • A2A: Enable seamless collaboration between AI agents across different platforms with LlmTornado.A2A.
  • Skills: Dynamically load folders of instructions, scripts, and resources to improve performance on specialized tasks.
• Vector Databases: Built-in connectors to Chroma, PgVector, Pinecone, Faiss, and QDrant.
• Integrated: Interoperability with Microsoft.Extensions.AI enables plugging Tornado in Semantic Kernel applications with LlmTornado.Microsoft.Extensions.AI.
• Enterprise Ready: Guardrails framework, preview and transform any request before committing to it. Open Telemetry support. Stable APIs.

News 2026

• 26/02 - MiniMax connector is implemented. Compaction endpoint is added, and can be used along the compaction built-in to LLM Tornado. Started work on ACP.
• 26/01 - Ivy Framework uses LlmTornado.Agents to build their AI Components. /ocr endpoint is implemented for Mistral. /files endpoint support is extended to all supported providers.

News 2025

• 25/12 - Upstage connector is implemented. /batch endpoint is implemented for OpenAI, Anthropic, and Google. /videos endpoint now supports OpenAI/Sora.
• 25/11 - Flowbite Blazor uses LLM Tornado to build their AI Chat WASM component. Requesty connector is implemented. /tokenize endpoint is implemented for all Providers supporting the feature.
• 25/10 - LLM Tornado is featured in dotInsights by JetBrains. Microsoft uses LLM Tornado in Generative AI for Beginners .NET. Interoperability with Microsoft.Extensions.AI is launched. Skills protocol is implemented.
• 25/09 - Maintainers Matěj Štágl and John Lomba talk about LLM Tornado in .NET AI Community Standup. PgVector and ChromaDb connectors are implemented.
• 25/08 - ProseFlow is built with LLM Tornado. Sciobot – an AI platform for Educators built with LLM Tornado is accepted into Cohere Labs Catalyst Grant Program. A2A protocol is implemented.
• 25/07 - Contributor Shaltiel Shmidman talks about LLM Tornado in Asp.net Community Standup. New connectors to Z.ai, Alibaba, and Blablador are implemented.
• 25/06 - C# delegates as Agent tools system is added, freeing applications from authoring JSON schema manually. MCP protocol is implemented.
• 25/05 - Chat to Responses system is added, allowing usage of /responses exclusive models from /chat endpoint.
• 25/04 - New connectors to DeepInfra, DeepSeek, and Perplexity are added.
• 25/03 - Assistants are implemented.
• 25/02 - New connectors to OpenRouter, Voyage, and xAI are added.
• 25/01 - Strict JSON mode is implemented, Groq and Mistral connectors are added.

⭐ Samples

• Chat with your documents
• Make multiple-speaker podcasts
• Voice call with AI using your microphone
• Orchestrate Assistants
• Generate images
• Summarize a video (local file / YouTube)
• Turn text & images into high quality embeddings
• Transcribe audio in real time

⚡Getting Started

Install LLM Tornado via NuGet:

dotnet add package LlmTornado

Optional addons:

dotnet add package LlmTornado.Agents # Agentic framework, higher-level abstractions
dotnet add package LlmTornado.Mcp # Model Context Protocol (MCP) integration
dotnet add package LlmTornado.A2A # Agent2Agent (A2A) integration
dotnet add package LlmTornado.Microsoft.Extensions.AI # Semantic Kernel interoperability
dotnet add package LlmTornado.Contrib # productivity, quality of life enhancements

Quick Inference

Inferencing across multiple providers is as easy as changing the ChatModel argument. Tornado instance can be constructed with multiple API keys, the correct key is then used based on the model automatically:

…

Instead of passing in a strongly typed model, you can pass a string instead: await api.Chat.CreateConversation("gpt-5-mini"), Tornado will automatically resolve the provider.

OpenAI API keys and Codex subscriptions

OpenAI API keys continue to support normal text and image models, while ChatGPT subscription access for Codex is explained in the Codex guide.

❄️ Vendor Extensions

Tornado has a powerful concept of VendorExtensions which can be applied to various endpoints and are strongly typed. Many Providers offer unique/niche APIs, often enabling use cases otherwise unavailable. For example, let's set a reasoning budget for Anthropic's Claude 3.7:

…

Self-Hosted/Custom Providers

Instead of consuming commercial APIs, one can easily roll their inference servers with a plethora of available tools. Here is a simple demo for streaming response with Ollama, but the same approach can be used for any custom provider:

…

https://github.com/user-attachments/assets/de62f0fe-93e0-448c-81d0-8ab7447ad780

Advanced Inference

Streaming

Tornado offers three levels of abstraction, trading more details for more complexity. The simple use cases where only plaintext is needed can be represented in a terse format:

await api.Chat.CreateConversation(ChatModel.Anthropic.Claude3.Sonnet)
    .AppendSystemMessage("You are a fortune teller.")
    .AppendUserInput("What will my future bring?")
    .StreamResponse(Console.Write);

The levels of abstraction are:

• Response (string for chat, float[] for embeddings, etc.)
• ResponseRich (tools, modalities, metadata such as usage)
• ResponseRichSafe (same as level 2, guaranteed not to throw on network level, for example, if the provider returns an internal error or doesn't respond at all)

Streaming with Rich content (tools, images, audio..)

When plaintext is insufficient, switch to StreamResponseRich or GetResponseRich() APIs. Tools requested by the model can be resolved later and never returned to the model. This is useful in scenarios where we use the tools without intending to continue the conversation:

…

Too