dnhkng/GLaDOS

GLaDOS Personality Core

"Science isn't about asking why. It's about asking, 'Why not?'" — Cave Johnson

A voice assistant with GLaDOS’s custom voice, dry humour, camera observations, persistent memory and tools. Talk through a local microphone or the browser, watch the animated core respond, and inspect what each mind is doing.

The project began with a simple question: could we build the personality core from Portal? The current system combines a conversational model with independent cores for emotion, memory, vision, system health and research. Their observations feed a shared context, while an Autonomy Core decides when an update deserves a spoken response.

Discord · Sponsor · Web console guide · Architecture

https://github.com/user-attachments/assets/c22049e4-7fba-4e84-8667-2c6657a656a0

What it does

"We've both said a lot of things that you're going to regret" — GLaDOS

• Voice conversation: Gemma 4 E4B can receive audio directly. An extended profile uses Parakeet transcription with a different conversation model.
• GLaDOS speech and animation: spoken emotion directions move the browser avatar; a separate Emotion Core maintains pleasure, arousal and dominance.
• A functional web console: control microphone, voice and camera; inspect minds, context slots, routing decisions, saved memory and background tasks.
• Vision: E4B describes the current scene and recent changes using timestamped images. Observation timing adapts to motion; the default range is 2–5 seconds.
• Memory and research: retain conversation summaries and useful facts, retrieve relevant memories, and perform requested internet research.
• Tools: connect MCP services, including Home Assistant, and use native controls for preferences, memory, tasks and allowlisted system commands.
• Proactive responses: independent cores publish evidence for the Autonomy Core to review. User interactions take priority over new background inference.

Quick start

"The Enrichment Center is required to remind you that the Weighted Companion Cube cannot talk. In the event that it does talk The Enrichment Centre asks you to ignore its advice." — GLaDOS

You need Git and Python 3.12 or newer to run the installer. It creates a Python 3.12.8 environment, installs GLaDOS and downloads the local ONNX speech models. The conversation model runs in a separate server.

The reference setup uses a CUDA build of llama.cpp and Gemma 4 E4B. See the model setup and measured results for hardware and context capacity details; latency and memory use depend on your machine and enabled cores.

1. Install GLaDOS

git clone https://github.com/dnhkng/GLaDOS.git
cd GLaDOS
python scripts/install.py

The installer selects CUDA when its utility is available, an installed ROCm backend otherwise, or CPU. You can override this:

python scripts/install.py --backend cpu
python scripts/install.py --backend cuda
python scripts/install.py --backend amd --rocm-version 7.2.1

Choose the command for your machine; these are alternatives. GPU drivers and CUDA/ROCm must be installed separately. For AMD, follow the AMD setup guide: the installer supports Linux x86_64 with ROCm 7.1, 7.2 and 7.2.1, using matching MIGraphX wheels. AMD model performance still needs hardware validation. An installed OpenVINO provider is also retained by local audio provider selection; the installer does not supply an Intel runtime.

On Debian/Ubuntu, local microphone and speaker access may require:

sudo apt install libportaudio2

Windows users should install Python 3.12+ and, if a runtime DLL is missing, the Visual C++ Redistributable. The AMD installer path is Linux-only. macOS support remains experimental.

2. Start the conversation model

Install a Gemma 4-capable CUDA llama-server separately. Download these two files from ggml-org/gemma-4-E4B-it-GGUF into models/gemma4-benchmark/:

• gemma-4-E4B-it-Q4_0.gguf
• mmproj-gemma-4-E4B-it-Q8_0.gguf

These GGUF files are not downloaded by glados download. Checksums and the tested revision are in the E4B setup guide. Then, from the repository root, run:

bash scripts/run_llamacpp.sh

If the executable or models are elsewhere:

LLAMA_SERVER=/path/to/llama-server \
GLADOS_MODEL_DIR=/path/to/gemma-models \
bash scripts/run_llamacpp.sh

The launcher serves gemma-4-E4B at http://127.0.0.1:18080, with four inference slots and 16K context per slot. It is tuned for CUDA; AMD ONNX installation does not configure this separate server. Use an appropriate server build/configuration for other hardware, or choose the extended profile below.

3. Open the web console

In another terminal:

uv run --no-sync glados webapp --config configs/glados_webapp_config.yaml

Open http://127.0.0.1:8050/. This profile uses the host machine’s microphone and speakers by default. Browser microphone/speaker setup is described below.

Use --no-sync after installer-based setup to preserve its selected ONNX runtime, particularly AMD’s separately installed vendor wheel. Rerun the installer to update or switch the backend; it preserves the environment and replaces the old ONNX distribution before installing the selected one.

Choose a conversation profile

English is the default input language. Direct-audio mode does not load Parakeet. Optional written user transcripts are off by default and use Gemma, not a second ASR model. Enable them in Facility Settings → Voice input when you want a written record. Extended mode uses Parakeet transcripts as model input.

The extended profile ships with gemma4:e4b on an Ollama endpoint. Supply that model yourself or change llm_model, completion_url and the provider-specific request options. Use a non-thinking model, or its minimal/disabled reasoning setting, for responsive conversation.

uv run --no-sync glados webapp --config configs/glados_extended_config.yaml

For cloud/text backends, keep native_audio.enabled: false; do not send Gemma’s native audio request format to an API that does not support it. Vision has its own E4B endpoint and can remain independent of the conversation model.

Browser audio and camera

The console can use local devices or browser media. To use browser audio, create configs/browser_audio.yaml:

Glados:
  audio_io: websocket
  audio_io_options:
    server: 127.0.0.1
    port: 5051
    rooms: false

Layer it over the web console profile:

uv run --no-sync glados webapp \
  --config configs/glados_webapp_config.yaml \
  --config configs/browser_audio.yaml

Open the console and use its microphone/speaker controls, granting browser permission when prompted. The HTTP console uses port 8050; browser audio uses the separate WebSocket server on port 5051. Camera observations can be controlled from the console, with camera selection in Facility Settings.

Keep the default loopback binding for local use. Remote browser media needs a secure browser context and suitable HTTP/WebSocket proxy configuration. A wildcard HTTP bind also requires explicit webapp.allowed_hosts; this allowlist is not authentication. See the web console guide for remote access, device permissions and configuration.

Using the console

The top bar switches between the operator console and a Presence screen for tablets. Both use the same live engine state, controls and eye animation. Choose Dark, Clinical, Terminal or PotatOS in the top bar or Facility Settings; the theme is remembered by this browser.

• Central Core / Brainstem: control input, spoken output, camera observations and Quiet mode. Quiet pauses replies and background cores until a wake request or the Wake control; disabling autonomy only disables proactive responses.
• Cores: inspect and control the workers that produce observations and results, including their shared inference capacity and waiting requests. Context slots hold stored information; they are separate from inference capacity.
• Test Chamber: browse saved facts and historical summaries.
• Neural Buffer: inspect the actual request context and its sources. The live system clock is included in reply context.
• Facility Settings: edit response instructions, routing choices and thresholds, preferred search sources, vision timing and optional transcripts.

Settings edited in the console persist as YAML under data/, including operator_settings.yaml, decision_lists.yaml, search_settings.yaml and vision_settings.yaml. Console edits apply to the running app; manual file edits take effect after restart. Memories and summaries have their own persistent stores. See memory and console controls for details.

Other launch modes

Run commands from the repository root:

uv run --no-sync glados                         # Local voice mode
uv run --no-sync glados tui                     # Terminal interface
uv run --no-sync glados start --input-mode text # Typed input
uv run --no-sync glados start --input-mode both # Voice and typed input
uv run --no-sync glados say "The cake is a lie" # Speech synthesis only
uv run --no-sync glados download               # Download/check local ONNX models

The TUI is an alternative to the web console. Use Ctrl+P for its command palette and F1 for help. glados --help and each command’s --help list available configuration and input/output overrides.

The model downloader currently verifies all registered local ONNX models, including optional ASR and Kokoro models. Downloading them does not mean they are all loaded: direct-audio mode skips Parakeet at runtime.

Configuration

"As part of a required test protocol, we will not monitor the next test chamber. You will be entirely on your own. Good luck." — GLaDOS

Configuration files contain a top-level Glados: mapping. Repeat --config to layer files; values in later files override earlier values. Start with a complete shipped profile and add small overlays for your changes.

For example, configs/personal.yaml:

Glados:
  voice: glados # Or a supported Kokoro voice such as af_bella.
  personality_preprompt:
    - system: "You are GLaDOS. Give useful, accurate answers with dry humour."
    - user: "What do you think of my code?"
    - assistant: "It runs. We should preserve this moment for the historians."

uv run --no-sync glados webapp \
  --config configs/glados_webapp_config.yaml \
  --config configs/personal.yaml

For a different text conversation backend, layer an override on the extended profile and set llm_model, completion_url, optional api_key/llm_headers, and the model’s supported llm_request_options. The included MiniMax configuration is another provider example; check your provider’s current model names and request options before using it.

How the cores work

"Let's be honest. Neither one of us knows what that thing does. Just put it in the corner and I'll deal with it later." — GLaDOS

flowchart LR
    mic[Microphone] --> vad[VAD and utterance buffer]
    vad --> input[Direct audio or Parakeet transcript]
    text[Typed input] --> route[Capability routing]
    input --> route
    route --> central[Central Core]
    route --> tools[Native and MCP tools]
    cores[Emotion, Memory, Vision, System, Search] --> slots[Context slots]
    tools --> slots
    slots --> central
    slots --> autonomy[Autonomy Core review]
    autonomy -->|Relevant update| central
    central --> tts[GLaDOS or Kokoro TTS]
    tts --> output[Speaker and animated avatar]

Core components

"All these science spheres are made out of asbestos, by the way. Keeps out the rats. Let us know if you feel a shortness of breath, a persistent dry cough, or your heart stopping. Because that's not part of the test. That's asbestos." — Cave Johnson

Independent cores publish regular updates, important updates and task results. The Autonomy Core considers their evidence together with the conversation before asking Central to speak. A completed search or recalled fact can be useful without needing another spoken response if the conversation already covers it.

Background minds share a timer and worker pool, with fixed, adaptive, random adaptive or on-demand timing assigned separately from their work. They can use ordinary code or services without an LLM. See mind scheduling.

Inference is admitted through a shared, bounded scheduler. The reference profile uses four server slots, with two reserved for interaction and routing. New background work waits during user interactions; already-running inference may finish, and stale outputs are rejected. Routing stays available. This reduces competition for the GPU without creating a separate model for each mind.

With llama.cpp, capability routing scores a small fixed set of token options. It can choose a reply, an intentional silence or an available action. Text replies may be drafted in parallel when capacity is free; direct-audio drafts remain disabled. Model-server capabilities determine which optimizations apply.

Jev-style routing during speech input

Jev-style routing uses the conversation model as a constrained classifier, rather than loading a separate routing model. Each decision stage assigns single-token letters such as A, B and C to the available choices. A compatible llama.cpp server returns their probabilities after softmax; GLaDOS normalizes the option scores and checks both confidence and the margin over the next choice. A tool route can require several stages: capability, service, then a saved action.

flowchart TD
    mic["Microphone audio"] --> vad["VAD buffers speech<br/>416 ms silence completes a turn"]
    vad --> mode{"Voice input mode"}
    mode -->|Default| audio["E4B direct audio<br/>No transcript required"]
    mode -->|Extended| asr["Parakeet transcript"]
    audio --> router["Jev-style routing on the conversation model<br/>Stable instructions and option list first<br/>Recent context and current input last"]
    asr --> router
    asr -. "When a spare slot is available" .-> draft["Speculative Central Core draft<br/>Runs alongside routing"]
    router --> scores["One generated option token per stage<br/>Option probabilities from llama.cpp"]
    scores --> gate{"Confidence and margin sufficient?"}
    gate -->|No| assist["Cancel draft; Central interprets the request<br/>No action authorized by classifier"]
    gate -->|Yes| choice{"Selected route"}
    choice -->|Ignore or quiet| silence["No spoken reply<br/>Cancel any speculative draft"]
    choice -->|Reply| reply["Central Core answers<br/>Reuse a compatible draft, otherwise generate"]
    choice -->|Capability or action| tool["Cancel draft; refine route if needed<br/>Execute validated saved tool arguments<br/>or ask Central to plan within tool scope"]
    tool --> result["Tool result enters context"]
    result --> reply
    assist --> reply
    draft -. "Used only for a compatible reply route" .-> reply
    reply --> tts["Stream speech chunks to TTS<br/>Emotion markers drive the avatar"]
    tts --> output["Speaker playback"]
    resume["User resumes before reply delivery"] -.-> cancel["Invalidate pending reply<br/>Extend the captured utterance"]
    cancel -.-> vad
    cancel -. "Cancel stale routing, draft and response" .-> router

The one-token limit applies to each routing decision's output. Audio encoding and prompt processing still take time. Direct-audio turns currently route before Central generates a reply; only transcript-based turns can overlap a speculative draft. Background cores yield new inference work during the interaction, while the Routing Core keeps its reserved access to the shared scheduler.

Speech capture uses 32 ms VAD chunks and a 416 ms silence gap. If the user resumes before the pending response begins delivery, the new speech can extend that turn and invalidate the pending response. Interruption also stops playback when interruptible is enabled. These timings do not imply a fixed end-to-end latency; benchmarks and caching details describe the measured setup.

Emotion markers such as [emotion:neutral] direct the avatar while speech is chunked for synthesis. They do not set the Emotion Core’s PAD state. Vision uses E4B scene observations and timestamped recent frames; YuNet face tracking uses OpenCV separately. The old FastVLM backend is no longer part of the app.

More detail: autonomy, vision, web console and routing, inference investigation.

Tools and integrations

Native tools include memory management, user preferences, task cancellation, slot management, reports, camera look requests and a fixed set of safe system commands. They are not a general-purpose shell interface.

MCP adds external capabilities through stdio, HTTP or SSE. For example, an overlay can register the bundled system-information server:

Glados:
  mcp_servers:
    - name: system_info
      transport: stdio
      command: python
      args: ["-m", "glados.mcp.system_info_server"]

An mcp_servers override replaces that list, so include any other servers you want to retain. The shipped profiles include requested internet search through Exa; this contacts an external service when research is used. Home Assistant requires your own server and credentials. See MCP configuration for tool filtering, memory services and transport setup.

GLaDOS speech API

"I'm speaking in an accent that is beyond her range of hearing." — Wheatley

The optional API exposes the GLaDOS voice through an OpenAI-style POST /v1/audio/speech endpoint. Other API voices and speed adjustment are not implemented. This endpoint does not require the conversation model server.

python scripts/install.py --api
uv run --no-sync litestar --app glados.api.app:create_app run \
  --host 127.0.0.1 --port 5050

For AMD, retain the same --backend amd --rocm-version ... options when running the installer with --api.

curl -X POST http://127.0.0.1:5050/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input":"Hello, test subject.","voice":"glados","response_format":"mp3"}' \
  --output speech.mp3

The API supports MP3, WAV and OGG output. It reuses the synthesizer by default; set Api.reuse_tts: false in api_config.yaml or use GLADOS_API_REUSE_TTS=false to load it per request.

A Docker API setup is also supplied:

docker compose up -d --build

This serves port 5050. It is the speech API, not the browser console or an installer for the E4B model server.

Troubleshooting

"No one will blame you for giving up. In fact, quitting at this point is a perfectly reasonable response." — GLaDOS

Development and next steps

"Federal regulations require me to warn you that this next test chamber... is looking pretty good." — GLaDOS

For a separate CPU development environment:

uv sync --extra cpu --extra dev
uv run --no-sync pytest -q tests

The API tests require the api extra and downloaded speech models. Do not use this CPU sync command to update an AMD environment; it replaces the selected runtime. Installer-based environments can add development tools with uv pip install -e ".[dev]" instead.

Recorded tests and experiments are in docs/benchmarks, with runnable probes in examples. The refactor roadmap records remaining runtime work; roadmap.md covers broader directions. AMD model validation, streaming ASR and physical animatronics remain future work.

Community

Questions, experiments and feedback are welcome on Discord. You can also sponsor development.

GLaDOS and Portal belong to Valve. This is a fan project.