helicone

Docs • Changelog • Bug reports • See Helicone in Action! (Free)

## Helicone is an AI Gateway & LLM Observability Platform for AI Engineers - **AI Gateway**: Access 100+ AI models with 1 API key through the OpenAI API with intelligent routing and automatic fallbacks. [Get started in 2 minutes.](https://docs.helicone.ai/gateway/overview) - **Quick integration**: One-line of code to log all your requests from [OpenAI](https://www.helicone.ai/models?providers=openai), [Anthropic](https://www.helicone.ai/models?providers=anthropic), [LangChain](https://docs.helicone.ai/gateway/integrations/langchain), [Gemini](https://www.helicone.ai/models?providers=gemini%2Cgoogle-ai-studio), [Vercel AI SDK](https://docs.helicone.ai/gateway/integrations/vercel-ai-sdk), and [more](https://docs.helicone.ai/gateway/overview). - **Observe**: Inspect and debug traces & [sessions](https://docs.helicone.ai/features/sessions) for agents, chatbots, document processing pipelines, and more - **Analyze**: Track metrics like [cost](https://docs.helicone.ai/faq/how-we-calculate-cost#developer), latency, quality, and more. Export to [PostHog](https://docs.helicone.ai/getting-started/integration-method/posthog) in one-line for custom dashboards - **Playground**: Rapidly test and iterate on prompts, sessions and traces in our UI. - **Prompt Management**: [Version prompts](https://docs.helicone.ai/features/prompts) using production data. Deploy prompts through the AI Gateway without code changes. Your prompts remain under your control, always accessible. - ️ **Fine-tune**: Fine-tune with one of our fine-tuning partners: [OpenPipe](https://openpipe.ai/) or [Autonomi](https://www.autonomi.ai/) (more coming soon) - ️ **Enterprise Ready**: SOC 2 and GDPR compliant > Generous monthly [free tier](https://www.helicone.ai/pricing) (10k requests/month) - No credit card required! > ## Quick Start ⚡️ 1. Get your API key by signing up [here](https://helicone.ai/signup) and add credits at [helicone.ai/credits](https://us.helicone.ai/credits) 2. Update the `baseURL` in your code and add your API key. ```typescript import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://ai-gateway.helicone.ai", apiKey: process.env.HELICONE_API_KEY, }); const response = await client.chat.completions.create({ model: "gpt-4o-mini", // claude-sonnet-4, gemini-2.0-flash or any model from https://www.helicone.ai/models messages: [{ role: "user", content: "Hello!" }] }); ``` 3. You're all set! View your logs at [Helicone](https://us.helicone.ai/dashboard) and access 100+ models through one API. ### Self-Hosting Open Source LLM Observability #### Docker Helicone is simple to self-host and update. To get started locally, just use our [docker-compose](https://docs.helicone.ai/getting-started/self-deploy-docker) file. ```bash # Clone the repository git clone https://github.com/Helicone/helicone.git cd docker cp .env.example .env # Start the services ./helicone-compose.sh helicone up ``` #### Helm For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at [email protected]. #### Manual (Not Recommended) Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions [here](https://docs.helicone.ai/getting-started/self-deploy). #### Architecture Helicone is comprised of five services: - **Web**: Frontend Platform (NextJS) - **Worker**: Proxy Logging (Cloudflare Workers) - **Jawn**: Dedicated Server for serving collecting logs (Express + Tsoa) - **Supabase**: Application Database and Auth - **ClickHouse**: Analytics Database - **Minio**: Object Storage for logs. ## Integrations ### Inference Providers | Integration | Supports | Description | | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | | AI Gateway | [JS/TS, Python, cURL](https://docs.helicone.ai/gateway/overview) | Unified API for 100+ providers with intelligent routing, automatic fallbacks, and unified observability | Async Logging (OpenLLMetry) | [JS/TS](https://docs.helicone.ai/getting-started/integration-method/openllmetry), [Python](https://www.npmjs.com/package/@helicone/helicone) | Asynchronous logging for multiple LLM platforms | | OpenAI | [JS/TS, Python](https://www.helicone.ai/models?providers=openai) | Inference provider | | Azure OpenAI | [JS/TS, Python](https://www.helicone.ai/models?providers=azure) | Inference provider | | Anthropic | [JS/TS, Python](https://www.helicone.ai/models?search=anthropic) | Inference provider | | Ollama | [JS/TS](https://docs.helicone.ai/integrations/ollama/javascript) | Run and use large language models locally | | AWS Bedrock | [JS/TS](https://www.helicone.ai/models?providers=azure%2Cbedrock) | Inference provider | | Gemini API | [JS/TS](https://www.helicone.ai/models?providers=google-ai-studio) | Inference provider | | Gemini Vertex AI | [JS/TS](https://www.helicone.ai/models?providers=vertex) | Gemini models on Google Cloud's Vertex AI | | Vercel AI | [JS/TS](https://docs.helicone.ai/gateway/integrations/vercel-ai-sdk) | AI SDK for building AI-powered applications | | Anyscale | [JS/TS, Python](https://www.helicone.ai/models?providers=anyscale) | Inference provider | | TogetherAI | [JS/TS, Python](https://www.helicone.ai/models?providers=together) | Inference provider | - | | Hyperbolic | [JS/TS, Python](https://www.helicone.ai/models?providers=hyperbolic) | Inference provider | High-performance AI inference platform | | Groq | [JS/TS, Python](https://www.helicone.ai/models?providers=groq) | High-performance models | | DeepInfra | [JS/TS, Python](https://www.helicone.ai/models?providers=deepinfra) | Serverless AI inference for various models | | | Fireworks AI | [JS/TS, Python](https://www.helicone.ai/models?providers=fireworks) | Fast inference API for open-source LLMs | ### Frameworks | Framework | Supports | Description | | --------------------------------------------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | LangChain | [JS/TS, Python](https://www.helicone.ai/models?providers=langchain) | Use AI Gateway with LangChain for unified provider access | | LlamaIndex | [Python](https://www.helicone.ai/models?providers=llamaindex) | Framework for building LLM-powered data applications | | LangGraph | [Python](https://www.helicone.ai/models?providers=langgraph) | Build stateful, multi-actor applications with LLMs | | Vercel AI SDK | [JS/TS](https://www.helicone.ai/models?providers=vercel-ai-sdk) | AI SDK for building AI-powered applications | | Semantic Kernel | [C#, Python](https://www.helicone.ai/models?providers=semantic-kernel) | Microsoft's AI orchestration framework | | CrewAI | [Python](https://docs.helicone.ai/integrations/openai/crewai) | Framework for orchestrating role-playing AI agents | | | ModelFusion | [JS/TS](https://github.com/vercel/modelfusion/blob/main/docs/integration/observability/helicone.md) | Abstraction layer for integrating AI models into JavaScript and TypeScript applications | | PostHog | [JS/TS, Python, cURL](https://docs.helicone.ai/getting-started/integration-method/posthog) | Product analytics platform. Build custom dashboards. | | RAGAS | [Python](https://docs.helicone.ai/other-integrations/ragas) | Evaluation framework for retrieval-augmented generation | | Open WebU