GLaDOS

GLaDOS Personality Core

Prologue

"Science isn't about asking why. It's about asking, 'Why not?'" - Cave Johnson

GLaDOS is the AI antagonist from Valve's Portal series—a sardonic, passive-aggressive superintelligence who views humans as test subjects worthy of both study and mockery.

Back in 2022 when ChatGPT made its debut, I had a realization: we are living in the Sci-Fi future and can actually build her now. A demented, obsessive AI fixated on humanity, super intelligent yet utterly lacking sound judgment; so just like an LLM, right? 2026, and still no moon colonies or flying cars. But a passive-aggressive AI that controls your lights and runs experiments on you? That we can do.

The architecture borrows from Minsky's Society of Mind—rather than one monolithic prompt, multiple specialized agents (vision, memory, personality, planning) each contribute to a dynamic context. GLaDOS's "self" emerges from their combined output, assembled fresh for each interaction.

The hard part was latency. Getting round-trip response time under 600 milliseconds is a threshold—below it, conversation stops feeling stilted and starts to flow. That meant training a custom TTS model and ruthlessly cutting milliseconds from every part of the pipeline.

Since 2023 I've refactored the system multiple times as better models came out. The current version finally adds what I always wanted: vision, memory, and tool use via MCP.

She sees through a camera, hears through a microphone, speaks through a speaker, and judges you accordingly.

Join our Discord! | Sponsor the project

https://github.com/user-attachments/assets/c22049e4-7fba-4e84-8667-2c6657a656a0

Vision

"We've both said a lot of things that you're going to regret" - GLaDOS

Most voice assistants wait for wake words. GLaDOS doesn't wait—she observes, thinks, and speaks when she has something to say. All the while, parts of her minds are tracking what she sees, monitoring system stats, and researching new neurotoxin recipes online.

Goals:

• Proactive behavior: React to events (vision, sound, time) without being prompted
• Emotional state: PAD model (Pleasure-Arousal-Dominance) for reactive mood
• Persistent personality: HEXACO traits provide stable character across sessions
• Multi-agent architecture: Subagents handle research, memory, emotions; main agent stays focused
• Real-time conversation: Optimized latency, natural interruption handling

What's New

• Emotions: PAD model for reactive mood + HEXACO traits for persistent personality
• Long-term Memory: Facts, preferences, and conversation summaries persist across sessions
• Observer Agent: Constitutional AI monitors behavior and self-adjusts within bounds
• Vision: FastVLM gives her eyes. Details | Demo
• Autonomy: She watches, waits, and speaks when she has something to say. Details
• MCP Tools: Extensible tool system for home automation, system info, etc. Details
• 8GB SBC: Runs on a Rock5b with RK3588 NPU. Branch

Roadmap

"Federal regulations require me to warn you that this next test chamber... is looking pretty good.” - GLaDOS

There's still a lot do do; I will be swapping out models are they are released, and then working on anamatronics, once a good model with inverse kinematics comes out. There was a time when I would code that myself; these days it makes more sense to wait until a trained model is released!

• Train GLaDOS voice
• Personality that actually sounds like her
• Vision via VLM
• Autonomy (proactive behavior)
• MCP tool system
• Emotional state (PAD + HEXACO model)
• Long-term memory
• Implement streaming ASR (nvidia/multitalker-parakeet-streaming-0.6b-v1)
• Observer agent (behavior adjustment)
• 3D-printable enclosure
• Animatronics

Architecture

"Let's be honest. Neither one of us knows what that thing does. Just put it in the corner and I'll deal with it later." - GLaDOS

…

GLaDOS runs a loop: each tick she reads her slots (weather, news, vision, mood), decides if she has something to say, and speaks. No wake word—if she has an opinion, you'll hear it.

Two lanes: Your speech jumps the queue (priority lane). The autonomy lane is just the loop running in the background. User always wins.

Audio Pipeline

…

• Microphone captures at 16kHz mono
• Silero VAD processes 32ms chunks, triggers at probability > 0.8
• Pre-activation buffer preserves 800ms before voice detected
• Silence detection waits 640ms pause before finalizing
• Interruption stops playback and clips the response in conversation history

Thread Architecture

Thread	Class	Daemon	Priority	Queue	Purpose
SpeechListener	SpeechListener	✓	INPUT	—	VAD + ASR
TextListener	TextListener	✓	INPUT	—	Text input
LLMProcessor	LanguageModelProcessor	✗	PROCESSING	llm_queue_priority	Main LLM
LLMProcessor-Auto-N	LanguageModelProcessor	✗	PROCESSING	llm_queue_autonomy	Autonomy LLM
ToolExecutor	ToolExecutor	✗	PROCESSING	tool_calls_queue	Tool execution
TTSSynthesizer	TextToSpeechSynthesizer	✗	OUTPUT	tts_queue	Voice synthesis
AudioPlayer	SpeechPlayer	✗	OUTPUT	audio_queue	Playback
AutonomyLoop	AutonomyLoop	✓	BACKGROUND	—	Tick orchestration
VisionProcessor	VisionProcessor	✓	BACKGROUND	vision_request_queue	Vision analysis

Daemon threads can be killed on exit. Non-daemon threads must complete gracefully to preserve state (e.g., conversation history).

Shutdown order: INPUT → PROCESSING → OUTPUT → BACKGROUND → CLEANUP

Context Building

…

What the LLM sees on each request:

1. System prompt with personality
2. Task slots (weather, news, vision state, emotion)
3. User preferences from memory
4. Constitutional modifiers (behavior adjustments from observer)
5. MCP resources (dynamic tool descriptions)
6. Conversation history (compacted when exceeding token threshold)

Autonomy System

…

Each subagent runs its own loop: timer or camera triggers it, it makes an LLM decision, and writes to a slot the main agent reads. Fully async—subagents never block the main conversation.

See autonomy.md for details.

Tool Execution

sequenceDiagram
    participant LLM
    participant Executor as Tool Executor
    participant MCP as MCP Server
    participant Native as Native Tool

    LLM->>Executor: tool_call {name, args}

    alt MCP Tool (mcp.*)
        Executor->>MCP: call_tool(server, tool, args)
        MCP-->>Executor: result
    else Native Tool
        Executor->>Native: run(tool_call_id, args)
        Native-->>Executor: result
    end

    Executor->>LLM: {role: tool, content: result}

Native tools: speak, do_nothing, get_user_preferences, set_user_preferences

MCP tools: Prefixed with server name (e.g., mcp.system_info.get_cpu). Supports stdio, HTTP, and SSE transports.

See mcp.md for configuration.

Components

"All these science spheres are made out of asbestos, by the way. Keeps out the rats. Let us know if you feel a shortness of breath, a persistent dry cough, or your heart stopping. Because that's not part of the test. That's asbestos." - Cave Johnson

Component	Technology	Purpose	Status
Speech Recognition	Parakeet TDT (ONNX)	Speech-to-text, 16kHz streaming	✅
Voice Activity	Silero VAD (ONNX)	Detect speech, 32ms chunks	✅
Voice Synthesis	Kokoro / GLaDOS TTS	Text-to-speech, streaming	✅
Interruption	VAD + Playback Control	Talk over her, she stops	✅
Vision	FastVLM (ONNX)	Scene understanding, change detection	✅
LLM	OpenAI-compatible API	Reasoning, tool use, streaming	✅
Tools	MCP Protocol	Extensibility, stdio/HTTP/SSE	✅
Autonomy	Subagent Architecture	Proactive behavior, tick loop	✅
Conversation	ConversationStore	Thread-safe history	✅
Compaction	LLM Summarization	Token management	✅
Emotional State	PAD + HEXACO	Reactive mood, persistent personality	✅
Long-term Memory	MCP + Subagent	Facts, preferences, summaries	✅
Observer Agent	Constitutional AI	Behavior adjustment	✅

✅ = Done | = In progress

Quick Start

"The Enrichment Center is required to remind you that the Weighted Companion Cube cannot talk. In the event that it does talk The Enrichment Centre asks you to ignore its advice." - GLaDOS

1. Install Ollama and grab a model:

   ollama pull llama3.2
   

2. Clone and install:

   git clone https://github.com/dnhkng/GLaDOS.git
   cd GLaDOS
   python scripts/install.py
   

3. Run:

   uv run glados          # Voice mode
   uv run glados tui      # Text interface
   

Installation

GPU Setup (recommended)

• NVIDIA: Install CUDA Toolkit
• AMD/Intel: Install appropriate ONNX Runtime

Works without GPU, just slower.

LLM Backend

GLaDOS needs an LLM. Options:

1. Ollama (easiest): ollama pull llama3.2
2. Any OpenAI-compatible API (OpenAI, MiniMax, OpenRouter, etc.)

Configure in glados_config.yaml:

completion_url: "http://localhost:11434/v1/chat/completions"
model: "llama3.2"
api_key: ""  # if needed

Cloud LLM Providers

You can use any OpenAI-compatible cloud API. Example configs are provided in configs/:

MiniMax — high-performance models with 512K context and built-in reasoning:

llm_model: "MiniMax-M3"
completion_url: "https://api.minimax.io/v1/chat/completions"
api_key: "your-minimax-api-key"

See configs/minimax_config.yaml for a complete configuration. Models: MiniMax-M3 (latest flagship, default), MiniMax-M2.7 (previous generation), MiniMax-M2.7-highspeed (low-latency).

OpenRouter — access multiple models through one API:

llm_model: "openai/gpt-4o"
completion_url: "https://openrouter.ai/api/v1/chat/completions"
api_key: "your-openrouter-api-key"
llm_headers:
  HTTP-Referer: "https://github.com/dnhkng/GLaDOS"
  X-Title: "GLaDOS"

Platform Notes

Linux:

sudo apt install libportaudio2

Windows: Install Python 3.12 from Microsoft Store.

macOS: Experimental. Check Discord for help.

Install

git clone https://github.com/dnhkng/GLaDOS.git
cd GLaDOS
python scripts/install.py

Usage

uv run glados                           # Voice mode
uv run glados tui                       # Text UI
uv run glados start --input-mode text   # Text only
uv run glados start --input-mode both   # Voice + text
uv run glados say "The cake is a lie"   # Just TTS

TUI Controls

Press Ctrl+P to open the command palette. Available commands:

Command	What it does
Status	System overview
Speech Recognition	Toggle ASR on/off
Text-to-Speech	Toggle TTS on/off
Config	View configuration
Memory	Long-ter