# GLaDOS Personality Core
## Prologue
> *"Science isn't about asking why. It's about asking, 'Why not?'" - Cave Johnson*
GLaDOS is the AI antagonist from Valve's Portal series—a sardonic, passive-aggressive superintelligence who views humans as test subjects worthy of both study and mockery.
Back in 2022 when ChatGPT made its debut, I had a realization: we are living in the Sci-Fi future and can actually build her now. A demented, obsessive AI fixated on humanity, super intelligent yet utterly lacking sound judgment; so just like an LLM, right? 2026, and still no moon colonies or flying cars. But a passive-aggressive AI that controls your lights and runs experiments on you? That we can do.
The architecture borrows from Minsky's Society of Mind—rather than one monolithic prompt, multiple specialized agents (vision, memory, personality, planning) each contribute to a dynamic context. GLaDOS's "self" emerges from their combined output, assembled fresh for each interaction.
The hard part was latency. Getting round-trip response time under 600 milliseconds is a threshold—below it, conversation stops feeling stilted and starts to flow. That meant training a custom TTS model and ruthlessly cutting milliseconds from every part of the pipeline.
Since 2023 I've refactored the system multiple times as better models came out. The current version finally adds what I always wanted: vision, memory, and tool use via MCP.
She sees through a camera, hears through a microphone, speaks through a speaker, and judges you accordingly.
[Join our Discord!](https://discord.com/invite/ERTDKwpjNB) | [Sponsor the project](https://ko-fi.com/dnhkng)
https://github.com/user-attachments/assets/c22049e4-7fba-4e84-8667-2c6657a656a0
## Vision
> *"We've both said a lot of things that you're going to regret" - GLaDOS*
Most voice assistants wait for wake words. GLaDOS doesn't wait—she observes, thinks, and speaks when she has something to say. All the while, parts of her minds are tracking what she sees, monitoring system stats, and researching new neurotoxin recipes online.
**Goals:**
- **Proactive behavior**: React to events (vision, sound, time) without being prompted
- **Emotional state**: PAD model (Pleasure-Arousal-Dominance) for reactive mood
- **Persistent personality**: HEXACO traits provide stable character across sessions
- **Multi-agent architecture**: Subagents handle research, memory, emotions; main agent stays focused
- **Real-time conversation**: Optimized latency, natural interruption handling
## What's New
- **Emotions**: PAD model for reactive mood + HEXACO traits for persistent personality
- **Long-term Memory**: Facts, preferences, and conversation summaries persist across sessions
- **Observer Agent**: Constitutional AI monitors behavior and self-adjusts within bounds
- **Vision**: FastVLM gives her eyes. [Details](/docs/vision.md) | [Demo](https://www.youtube.com/watch?v=JDd9Rc4toEo)
- **Autonomy**: She watches, waits, and speaks when she has something to say. [Details](/docs/autonomy.md)
- **MCP Tools**: Extensible tool system for home automation, system info, etc. [Details](/docs/mcp.md)
- **8GB SBC**: Runs on a Rock5b with RK3588 NPU. [Branch](https://github.com/dnhkng/RKLLM-Gradio)
## Roadmap
> *"Federal regulations require me to warn you that this next test chamber... is looking pretty good.” - GLaDOS*
There's still a lot do do; I will be swapping out models are they are released, and then working on anamatronics, once a good model with inverse kinematics comes out. There was a time when I would code that myself; these days it makes more sense to wait until a trained model is released!
- [x] Train GLaDOS voice
- [x] Personality that actually sounds like her
- [x] Vision via VLM
- [x] Autonomy (proactive behavior)
- [x] MCP tool system
- [x] Emotional state (PAD + HEXACO model)
- [x] Long-term memory
- [ ] Implement streaming ASR (nvidia/multitalker-parakeet-streaming-0.6b-v1)
- [ ] Observer agent (behavior adjustment)
- [ ] 3D-printable enclosure
- [ ] Animatronics
## Architecture
> *"Let's be honest. Neither one of us knows what that thing does. Just put it in the corner and I'll deal with it later." - GLaDOS*
```
…
```
GLaDOS runs a loop: each tick she reads her slots (weather, news, vision, mood), decides if she has something to say, and speaks. No wake word—if she has an opinion, you'll hear it.
**Two lanes**: Your speech jumps the queue (priority lane). The autonomy lane is just the loop running in the background. User always wins.
Audio Pipeline
```
…
```
- **Microphone** captures at 16kHz mono
- **Silero VAD** processes 32ms chunks, triggers at probability > 0.8
- **Pre-activation buffer** preserves 800ms before voice detected
- **Silence detection** waits 640ms pause before finalizing
- **Interruption** stops playback and clips the response in conversation history
Thread Architecture
| Thread | Class | Daemon | Priority | Queue | Purpose |
|--------|-------|--------|----------|-------|---------|
| SpeechListener | `SpeechListener` | ✓ | INPUT | — | VAD + ASR |
| TextListener | `TextListener` | ✓ | INPUT | — | Text input |
| LLMProcessor | `LanguageModelProcessor` | ✗ | PROCESSING | `llm_queue_priority` | Main LLM |
| LLMProcessor-Auto-N | `LanguageModelProcessor` | ✗ | PROCESSING | `llm_queue_autonomy` | Autonomy LLM |
| ToolExecutor | `ToolExecutor` | ✗ | PROCESSING | `tool_calls_queue` | Tool execution |
| TTSSynthesizer | `TextToSpeechSynthesizer` | ✗ | OUTPUT | `tts_queue` | Voice synthesis |
| AudioPlayer | `SpeechPlayer` | ✗ | OUTPUT | `audio_queue` | Playback |
| AutonomyLoop | `AutonomyLoop` | ✓ | BACKGROUND | — | Tick orchestration |
| VisionProcessor | `VisionProcessor` | ✓ | BACKGROUND | `vision_request_queue` | Vision analysis |
**Daemon threads** can be killed on exit. **Non-daemon threads** must complete gracefully to preserve state (e.g., conversation history).
**Shutdown order**: INPUT → PROCESSING → OUTPUT → BACKGROUND → CLEANUP
Context Building
```
…
```
What the LLM sees on each request:
1. **System prompt** with personality
2. **Task slots** (weather, news, vision state, emotion)
3. **User preferences** from memory
4. **Constitutional modifiers** (behavior adjustments from observer)
5. **MCP resources** (dynamic tool descriptions)
6. **Conversation history** (compacted when exceeding token threshold)
Autonomy System
```
…
```
Each subagent runs its own loop: timer or camera triggers it, it makes an LLM decision, and writes to a slot the main agent reads. Fully async—subagents never block the main conversation.
See [autonomy.md](/docs/autonomy.md) for details.
Tool Execution
```mermaid
sequenceDiagram
participant LLM
participant Executor as Tool Executor
participant MCP as MCP Server
participant Native as Native Tool
LLM->>Executor: tool_call {name, args}
alt MCP Tool (mcp.*)
Executor->>MCP: call_tool(server, tool, args)
MCP-->>Executor: result
else Native Tool
Executor->>Native: run(tool_call_id, args)
Native-->>Executor: result
end
Executor->>LLM: {role: tool, content: result}
```
**Native tools**: `speak`, `do_nothing`, `get_user_preferences`, `set_user_preferences`
**MCP tools**: Prefixed with server name (e.g., `mcp.system_info.get_cpu`). Supports stdio, HTTP, and SSE transports.
See [mcp.md](/docs/mcp.md) for configuration.
### Components
> *"All these science spheres are made out of asbestos, by the way. Keeps out the rats. Let us know if you feel a shortness of breath, a persistent dry cough, or your heart stopping. Because that's not part of the test. That's asbestos." - Cave Johnson*
| Component | Technology | Purpose | Status |
|-----------|------------|---------|--------|
| **Speech Recognition** | Parakeet TDT (ONNX) | Speech-to-text, 16kHz streaming | ✅ |
| **Voice Activity** | Silero VAD (ONNX) | Detect speech, 32ms chunks | ✅ |
| **Voice Synthesis** | Kokoro / GLaDOS TTS | Text-to-speech, streaming | ✅ |
| **Interruption** | VAD + Playback Control | Talk over her, she stops | ✅ |
| **Vision** | FastVLM (ONNX) | Scene understanding, change detection | ✅ |
| **LLM** | OpenAI-compatible API | Reasoning, tool use, streaming | ✅ |
| **Tools** | MCP Protocol | Extensibility, stdio/HTTP/SSE | ✅ |
| **Autonomy** | Subagent Architecture | Proactive behavior, tick loop | ✅ |
| **Conversation** | ConversationStore | Thread-safe history | ✅ |
| **Compaction** | LLM Summarization | Token management | ✅ |
| **Emotional State** | PAD + HEXACO | Reactive mood, persistent personality | ✅ |
| **Long-term Memory** | MCP + Subagent | Facts, preferences, summaries | ✅ |
| **Observer Agent** | Constitutional AI | Behavior adjustment | ✅ |
✅ = Done | = In progress
## Quick Start
> *"The Enrichment Center is required to remind you that the Weighted Companion Cube cannot talk. In the event that it does talk The Enrichment Centre asks you to ignore its advice." - GLaDOS*
1. Install [Ollama](https://github.com/ollama/ollama) and grab a model:
```bash
ollama pull llama3.2
```
2. Clone and install:
```bash
git clone https://github.com/dnhkng/GLaDOS.git
cd GLaDOS
python scripts/install.py
```
3. Run:
```bash
uv run glados # Voice mode
uv run glados tui # Text interface
```
## Installation
### GPU Setup (recommended)
- **NVIDIA**: Install [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)
- **AMD/Intel**: Install appropriate [ONNX Runtime](https://onnxruntime.ai/docs/install/)
Works without GPU, just slower.
### LLM Backend
GLaDOS needs an LLM. Options:
1. [Ollama](https://github.com/ollama/ollama) (easiest): `ollama pull llama3.2`
2. Any OpenAI-compatible API (OpenAI, [MiniMax](https://platform.minimaxi.com/), OpenRouter, etc.)
Configure in `glados_config.yaml`:
```yaml
completion_url: "http://localhost:11434/v1/chat/completions"
model: "llama3.2"
api_key: "" # if needed
```
#### Cloud LLM Providers
You can use any OpenAI-compatible cloud API. Example configs are provided in `configs/`:
**MiniMax** — high-performance models with 512K context and built-in reasoning:
```yaml
llm_model: "MiniMax-M3"
completion_url: "https://api.minimax.io/v1/chat/completions"
api_key: "your-minimax-api-key"
```
See `configs/minimax_config.yaml` for a complete configuration. Models: `MiniMax-M3` (latest flagship, default), `MiniMax-M2.7` (previous generation), `MiniMax-M2.7-highspeed` (low-latency).
**OpenRouter** — access multiple models through one API:
```yaml
llm_model: "openai/gpt-4o"
completion_url: "https://openrouter.ai/api/v1/chat/completions"
api_key: "your-openrouter-api-key"
llm_headers:
HTTP-Referer: "https://github.com/dnhkng/GLaDOS"
X-Title: "GLaDOS"
```
### Platform Notes
**Linux:**
```bash
sudo apt install libportaudio2
```
**Windows:**
Install Python 3.12 from Microsoft Store.
**macOS:**
Experimental. Check Discord for help.
### Install
```bash
git clone https://github.com/dnhkng/GLaDOS.git
cd GLaDOS
python scripts/install.py
```
## Usage
```bash
uv run glados # Voice mode
uv run glados tui # Text UI
uv run glados start --input-mode text # Text only
uv run glados start --input-mode both # Voice + text
uv run glados say "The cake is a lie" # Just TTS
```
### TUI Controls
Press `Ctrl+P` to open the command palette. Available commands:
| Command | What it does |
|---------|-------------|
| Status | System overview |
| Speech Recognition | Toggle ASR on/off |
| Text-to-Speech | Toggle TTS on/off |
| Config | View configuration |
| Memory | Long-ter