fastrtc

The Real-Time Communication Library for Python.

Turn any python function into a real-time audio and video stream over WebRTC or WebSockets. ## Installation ```bash pip install fastrtc ``` to use built-in pause detection (see [ReplyOnPause](https://fastrtc.org/userguide/audio/#reply-on-pause)), and text to speech (see [Text To Speech](https://fastrtc.org/userguide/audio/#text-to-speech)), install the `vad` and `tts` extras: ```bash pip install "fastrtc[vad, tts]" ``` ## Key Features - ️ Automatic Voice Detection and Turn Taking built-in, only worry about the logic for responding to the user. - Automatic UI - Use the `.ui.launch()` method to launch the webRTC-enabled built-in Gradio UI. - Automatic WebRTC Support - Use the `.mount(app)` method to mount the stream on a FastAPI app and get a webRTC endpoint for your own frontend! - ⚡️ Websocket Support - Use the `.mount(app)` method to mount the stream on a FastAPI app and get a websocket endpoint for your own frontend! - Automatic Telephone Support - Use the `fastphone()` method of the stream to launch the application and get a free temporary phone number! - Completely customizable backend - A `Stream` can easily be mounted on a FastAPI app so you can easily extend it to fit your production application. See the [Talk To Claude](https://huggingface.co/spaces/fastrtc/talk-to-claude) demo for an example of how to serve a custom JS frontend. ## Docs [https://fastrtc.org](https://fastrtc.org) ## Examples See the [Cookbook](https://fastrtc.org/cookbook/) for examples of how to use the library.

️ Gemini Audio Video Chat

Stream BOTH your webcam video and audio feeds to Google Gemini. You can also upload images to augment your conversation!

Demo | Code

️ Google Gemini Real Time Voice API

Talk to Gemini in real time using Google's voice API.

Demo | Code

️ OpenAI Real Time Voice API

Talk to ChatGPT in real time using OpenAI's voice API.

Demo | Code

Hello Computer

Say computer before asking your question!

Demo | Code

Llama Code Editor

Create and edit HTML pages with just your voice! Powered by SambaNova systems.

Demo | Code

️ Talk to Claude

Use the Anthropic and Play.Ht APIs to have an audio conversation with Claude.

Demo | Code

Whisper Transcription

Have whisper transcribe your speech in real time!

Demo | Code

Yolov10 Object Detection

Run the Yolov10 model on a user webcam stream in real time!

Demo | Code

️ Kyutai Moshi

Kyutai's moshi is a novel speech-to-speech model for modeling human conversations.

Demo | Code

️ Hello Llama: Stop Word Detection

A code editor built with Llama 3.3 70b that is triggered by the phrase "Hello Llama". Build a Siri-like coding assistant in 100 lines of code!

Demo | Code

## Usage This is a shortened version of the official [usage guide](https://freddyaboulton.github.io/gradio-webrtc/user-guide/). - `.ui.launch()`: Launch a built-in UI for easily testing and sharing your stream. Built with [Gradio](https://www.gradio.app/). - `.fastphone()`: Get a free temporary phone number to call into your stream. Hugging Face token required. - `.mount(app)`: Mount the stream on a [FastAPI](https://fastapi.tiangolo.com/) app. Perfect for integrating with your already existing production system. ## Quickstart ### Echo Audio ```python from fastrtc import Stream, ReplyOnPause import numpy as np def echo(audio: tuple[int, np.ndarray]): # The function will be passed the audio until the user pauses # Implement any iterator that yields audio # See "LLM Voice Chat" for a more complete example yield audio stream = Stream( handler=ReplyOnPause(echo), modality="audio", mode="send-receive", ) ``` ### LLM Voice Chat ``` … ``` ### Webcam Stream ```python from fastrtc import Stream import numpy as np def flip_vertically(image): return np.flip(image, axis=0) stream = Stream( handler=flip_vertically, modality="video", mode="send-receive", ) ``` ### Object Detection ``` … ``` ## Running the Stream Run: ### Gradio ```py stream.ui.launch() ``` ### Telephone (Audio Only) ```py stream.fastphone() ``` ### FastAPI ```py app = FastAPI() stream.mount(app) # Optional: Add routes @app.get("/") async def _(): return HTMLResponse(content=open("index.html").read()) # uvicorn app:app --host 0.0.0.0 --port 8000 ```