jarvis

Open source

A 100% private AI voice assistant that lives on your computer (works offline). Talk naturally as if Jarvis is a third person in the room, and get conversational

1.5K stars0 likes2 views

Website GitHub

About

A 100% private AI voice assistant that lives on your computer (works offline). Talk naturally as if Jarvis is a third person in the room, and get conversational

Jarvis

A 100% private AI voice assistant that lives on your computer (works offline). Talk naturally as if Jarvis is a third person in the room — say its name anywhere in your sentence and get conversational, context-aware responses. It remembers everything, always knows the current location and time, can search the web, read your screen, control Chrome, track nutrition, and much more with support for unlimited MCPs and tools without context rot. Sensitive info is automatically redacted before anything is saved to disk.

100% local processing. No subscriptions. No data harvesting. Automatic redaction of sensitive info. Free offline dictation included.

Support Jarvis

Why Jarvis?

** Your data stays yours** - 100% local AI processing. No cloud, no subscriptions, no data harvesting. Automatic redaction of sensitive info. This is non-negotiable.

️ A third person in the room - Unlike voice assistants that only respond to rigid commands, Jarvis understands conversations. It maintains a short temporary rolling context of what's being discussed, so when you ask "Jarvis, what do you think?" it knows exactly what you're talking about. Have it chime into discussions with friends, help debug code while you talk through problems, or weigh in on decisions.

** Never forgets** - Unlimited memory across conversations. Adapts tone naturally to the topic. Learns your preferences over time.

️ Free dictation - Hold a hotkey, speak, release — your words appear in any app as text. Like WisprFlow, but free, offline, and private. No subscription, no cloud transcription.

** Extensible** - MCP integration connects Jarvis to thousands of tools: smart home, GitHub, Slack, databases, and more. Smart tool selection means adding more tools won't slow things down.

** Transparent progress** - We track what works (and what doesn't) with automated evals. See current accuracy →

** Known limitations:** Jarvis is under active development. Primary development happens on macOS. Windows/Linux support may lag behind. We're building in the open, issues and contributions welcome!

Voice-only for now—no text chat interface yet (#35)
No mobile apps (#17)
"Stop" commands during speech sometimes get filtered as echo (#24)
Dictation is not available on macOS 26+ (Tahoe) due to a pynput incompatibility (#172)

See it in action (example conversations)

Chiming into conversations (the magic moment):

 Alice: I wonder what the weather will be like tomorrow
 Bob: Yeah, we should check before planning the picnic
 Alice: Jarvis, what do you think?
   Heard: "What do you think Jarvis?"
   Intent (wake word): directed → "what do you think about the weather for the picnic"

✨ Working on it: what do you think about the weather for the picnic
   Tool: getWeather…
   Generating response...

 Jarvis
Tomorrow looks great for a picnic! Sunny with highs around 22°C...

Jarvis understood the entire conversation and gave a contextual answer — no need to repeat the question.

Browser automation via MCP:

   Heard: "Open YouTube Jarvis."
   Intent (wake word): directed → "open YouTube"

✨ Working on it: open YouTube
   Generating response...

 Jarvis
I have opened YouTube for you.

Health-aware advice:

   Heard: "Should I order pizza tonight considering my health goals Jarvis?"
   Intent (wake word): directed → "should I order pizza tonight considering my health goals"

✨ Working on it: should I order pizza tonight considering my health goals
   Tool: fetchMeals…
   Generating response...

 Jarvis
Looking at your week — you mentioned wanting to lose 5kg by June...
Today you've had about 860 calories so far. Given your 1,800 calorie target,
you've got room for pizza! But maybe consider thin crust to save calories.

Personalised news search:

   Heard: "What are some news from today that might interest me Jarvis?"
   Intent (wake word): directed → "what are some news from today that might interest me"

✨ Working on it: what are some news from today that might interest me
   Tool: searchMemory…
   Tool: webSearch…
   Generating response...

 Jarvis
Here's a quick snapshot of today's headlines...

Natural wake word placement (say "Jarvis" anywhere):

   Heard: "Give me a random topic to discuss Jarvis."
   Intent (wake word): directed → "give me a random topic to discuss"

✨ Working on it: give me a random topic to discuss
   Generating response...

 Jarvis
How about the Fermi Paradox? Given the vast number of stars...

Echo detection (Jarvis ignores its own speech):

 Jarvis
I have opened YouTube for you.

 Listening for follow-up (3s)...
   Heard: "I have opened YouTube for you."
   Heard (echo): "i have opened youtube for you."
 Returning to wake word mode

Quick Install

1. Install Prerequisites

Platform	Requirement
All	Ollama (default), or any OpenAI-compatible server you already run (LM Studio, Jan, llama.cpp, vLLM, oMLX, LocalAI, …) — see Configuration → LLM Provider

2. Download Jarvis

Get the latest from GitHub Releases:

Platform	Download	Run
Windows	`Jarvis-Windows-x64.zip`	Extract → Run `Jarvis.exe`
macOS	`Jarvis-macOS-arm64.zip`	Extract → Move to Applications → Right-click → Open
Linux	`Jarvis-Linux-x64.tar.gz`	`tar -xzf` → Run `./Jarvis/Jarvis`

Jarvis starts listening automatically — just say "Jarvis" and talk!

Features

Conversational Awareness - Understands ongoing discussions. Ask "Jarvis, what do you think?" and it knows what you're talking about. Works naturally in multi-person conversations.
Unlimited Memory - Never forgets. Searches across all your conversation history. Memory Viewer GUI included.
Adaptive Tone - Automatically surgical for code, pragmatic for business, encouraging for wellbeing — no manual mode switching
Smart Tool Selection - Embedding-based relevance filtering picks only the tools needed per query — add unlimited MCP tools without performance degradation
Built-in Tools - Screenshot OCR, web search (DuckDuckGo → Brave → Wikipedia fallback chain with auto-fetch), weather, file access, nutrition tracking, location awareness, plus a tool-discovery escape hatch the agent uses to widen its own toolset mid-reply
Knowledge Graph Memory - Self-organising memory that learns from conversations, auto-splits by topic, and surfaces relevant knowledge automatically
Natural Voice - Say "Jarvis" anywhere in your sentence, interrupt with "stop", follow up without repeating the wake word
Dictation Mode - Free, offline alternative to WisprFlow — hold a hotkey, speak, release to paste text into any app
MCP Integration - Connect to thousands of external tools (Home Assistant, GitHub, Slack, etc.)

System Requirements

Hardware	VRAM	Model
Low-VRAM / CPU	2GB+	`qwen3.5:0.8b`
Most users	8GB+	`gemma4:e2b` (default)
Better quality	16GB+	`gemma4:e4b`
High-end	24GB+	`gpt-oss:20b`

Note: VRAM requirements include the fast model (gemma4:e2b) which is always loaded alongside the chat model for voice intent classification and other real-time work. The default chat model shares this, so no extra VRAM is needed.

The setup wizard will guide you through model selection and installation on first launch.

Configuration

Most users won't need to change anything. Open ⚙️ Settings from the tray menu to configure Jarvis through a graphical interface — no JSON editing required. Settings are saved to ~/.config/jarvis/config.json.

LLM Provider (Ollama or OpenAI-compatible)

By default Jarvis runs everything locally through Ollama: no API keys, nothing leaves your machine. If you already run an OpenAI-compatible server you can point Jarvis at it instead. Your data still only travels to the servers you control.

Pick the provider in the Setup Wizard's first step, or under ⚙️ Settings → LLM Provider. No JSON editing required. On the OpenAI-compatible page the wizard does the legwork for you: it auto-detects running local servers, offers a one-click preset for your app, and when you press Connect it loads the server's model list and checks the chosen model for chat, tool calling, and embeddings, so you know it works before you finish setup.

Tested local servers (all run on your own machine):

App	Default base URL	Notes
LM Studio	`http://localhost:1234/v1`	Chat, tool calling, and embeddings.
Ollama (OpenAI API)	`http://localhost:11434/v1`	The native Ollama path is the default; the OpenAI shape works too.
Jan	`http://localhost:1337/v1`	Chat and tool calling.
llama.cpp (`llama-server`)	`http://localhost:8080/v1`	Tool calling depends on the model.
LocalAI	`http://localhost:8080/v1`	Feature support depends on the backend model.
vLLM	`http://localhost:8000/v1`	Tool calling depends on the model.
oMLX (Apple Silicon)	varies	No embeddings endpoint, so memory uses keyword search unless you route embeddings to Ollama (below).

For reference, the underlying config keys are:

json

{
  "llm_provider": "openai_compatible",
  "llm_base_url": "http://localhost:1234/v1",
  "llm_api_key": "",
  "llm_chat_model": "your-served-model-name"
}

llm_base_url: your server's OpenAI API base URL.
llm_api_key: only if your server requires one; leave empty otherwise.
llm_chat_model: whatever model name your server exposes.
fast_model (optional): the small, quick model used for real-time work (voice intent, tool routing, quick classifications). Leave empty for automatic: gemma4:e2b on Ollama, your chat model on an OpenAI-compatible server. Set it to pin a dedicated small model.

Embeddings (used for memory search) can run on a different backend. If your chat server has no embeddings endpoint, memory falls back to keyword search. To keep full semantic memory, route embeddings to Ollama (the wizard offers this automatically when it detects a server that cannot embed):

json

{
  "embedding_provider": "ollama",
  "embedding_model": "nomic-embed-text"
}

Leave embedding_provider empty to use the same provider as chat. With no working embeddings, memory search degrades gracefully to keyword search.

Speech Recognition (Whisper)

Language Modes

Multilingual (default, 99 languages): "whisper_model": "medium"
English Only (slightly better English accuracy): "whisper_model": "medium.en"

Model Sizes

Model	English	Multilingual	Download	VRAM	Speed
Tiny	`tiny.en`	`tiny`	~75 MB	~1 GB	~10x
Base	`base.en`	`base`	~140 MB	~1 GB	~7x
Small	`small.en`	`small`	~465 MB	~2 GB	~4x
Medium	`medium.en`	`medium`	~1.5 GB	~5 GB	~2x
Large V3 Turbo	-	`large-v3-turbo`	~1.5 GB	~6 GB	~8x