Atomic Chat: Local AI App for Open-Weight LLMs
/Atomic-Chat
Atomic Chat is a desktop and mobile app that runs open-weight language models locally behind a single OpenAI-compatible API at localhost:1337/v1. It switches between three inference engines — llama.cpp, a TurboQuant fork of it, and Apple's MLX-VLM — without touching client code. Reach for it if you want offline AI chat with a cloud fallback in the same app; skip it if you just want one simple chat window.
What is Atomic Chat?
Atomic Chat is AtomicBot-ai's local AI app for running open-weight LLMs — Llama, Qwen, Mistral, Gemma, Phi, and others pulled from Hugging Face — entirely on your own hardware, paired with its own local inference engine. It ships as a desktop AI client for macOS, Windows, and Linux plus mobile apps on the App Store and Google Play, and its codebase traces back to an early fork of Jan before it diverged into its own set of inference engines.
Core Features and Capabilities
- ✓An OpenAI-compatible local server at http://localhost:1337/v1 that defaults to listening on 127.0.0.1 only, so any client using the OpenAI SDK can point at it after changing just the base URL.
- ✓Three swappable inference engines behind that one API: an in-house TurboQuant fork of llama.cpp, upstream llama.cpp, and Apple's MLX-VLM for vision-language models on Apple Silicon.
- ✓Speculative decoding options — Multi-Token Prediction (MTP) for a 30-70% throughput boost (up to 3x on Gemma 4), a DFlash block-diffusion mode that's up to 6x faster on models such as Qwen 3.6 and Kimi K2.5, and EAGLE-3 decoding for Gemma 4 on MLX.
- ✓TurboQuant KV-cache compression (turbo3/turbo4 modes) that the README says cuts KV-cache memory by roughly 4.3x, now working on Windows and Linux CPU/GPU as well as macOS.
- ✓Built-in cloud providers — Anthropic, OpenAI, Groq, MiniMax, Mistral, Qwen, and Moonshot — so you can bring your own API key and mix local and cloud models in the same chat.
- ✓One-click launches for coding agents including Claude Code, Codex CLI, Cline, OpenCode, Droid, Goose, OpenHands, Copilot CLI, Kilo Code, and Zed, straight from an Integrations tab.
- ✓MCP server connections for adding external tools, file access, or web search, plus custom per-assistant system prompts and a project sidebar with a conversation tree view.
- ✓An Artifacts panel that live-previews generated HTML/CSS/JS with copy, download, and print buttons.
Integrations and Agent Support
- •Running a coding agent such as Claude Code or Codex CLI against a local model, launched straight from Atomic Chat's Integrations tab instead of hand-editing a provider config file.
- •Pointing an existing OpenAI SDK-based app, script, or agent framework at http://localhost:1337/v1 to swap a cloud model for a local one without changing client code.
- •Wiring in MCP servers for extra tools, file access, or web search inside a chat, per the README's MCP support.
- •Using local providers already documented by other projects — OpenCode, Goose, nanobot, nanoclaw, OpenClaude, Kilo Code, Hermes Desktop, and Hermes Workspace all list Atomic Chat as a supported backend.
- •Running vision-language models on Apple Silicon through the MLX-VLM engine when you need faster inference on M-series chips than llama.cpp gives you, per the README.
Getting Started with Atomic Chat
The README points to atomic.chat for prebuilt desktop and mobile downloads, and to the App Store and Google Play for iOS and Android, but doesn't list exact download URLs or version numbers in the text itself. On Linux, Atomic Chat is distributed as one AppImage file, so there's nothing to install and no root access required — make it executable with `chmod +x Atomic.Chat_*_amd64.AppImage` and run it directly; if you're prompted about FUSE the first time, install it with `sudo apt install fuse libfuse2` on Debian/Ubuntu or `sudo dnf install fuse fuse-libs` on Fedora. To build from source you need Node.js 20.0.0 or newer, Yarn 4.5.3 or newer, Make 3.81 or newer, Rust for Tauri, and on Apple Silicon the MetalToolchain component; then `git clone https://github.com/AtomicBot-ai/Atomic-Chat`, `cd Atomic-Chat`, and `make dev` handles the dependency install, core build, and app launch in one step. Manual steps are also documented: `yarn install`, `yarn build:tauri:plugin:api`, `yarn build:core`, `yarn build:extensions`, `yarn dev`.
Strengths
- ✓One OpenAI-compatible endpoint in front of three different inference engines, so moving from upstream llama.cpp to the TurboQuant fork or MLX-VLM doesn't require touching client code.
- ✓Speculative decoding options (MTP, DFlash, EAGLE-3) tied to specific, sizeable speedups in the README — up to 3x on Gemma 4 with MTP, up to 6x on Qwen 3.6/Gemma 4/Kimi K2.5 with DFlash — rather than a vague faster-inference claim.
- ✓Coding-agent setup is a checkbox, not a config file: pick Claude Code, Codex CLI, or another supported agent from the Integrations tab and it launches pointed at your local model automatically.
- ✓Switching between a local model and a cloud one — OpenAI, Anthropic, or another built-in provider — happens with a per-chat model picker instead of a settings-file edit.
- ✓Apache 2.0 licensing, with no proprietary strings attached to the code.
System Requirements and Considerations
- △RAM scales with model size, per the README: about 8GB covers 3B models, 16GB covers 7B models, and 32GB covers 13B models, the same guidance across macOS, Windows, and Linux.
- △Linux only runs GGUF models, and GPU acceleration there depends on having a Vulkan loader installed — the README lists `vulkan-1` or the mesa/NVIDIA Vulkan drivers as an optional extra, not something bundled by default.
- △DFlash speculative decoding is Apple Silicon-only and can't be turned on at the same time as MTP, so Windows and Linux users don't get every decoding speedup mode.
- △The OS floor is fairly specific: Windows needs a 64-bit 10/11 install, and Linux needs glibc 2.35 or newer (Ubuntu 22.04+, Debian 12+, Fedora 40+, Arch, Mint, or Pop!_OS) — older distros or 32-bit systems aren't listed as supported.
- △The README doesn't state a version number or release cadence, so there's no documented way to gauge how mature the current build is beyond the star and contributor counts. It's also a young project run by a small core team (140+ contributors per the README) building on Jan's original codebase, so a long-term maintenance track record isn't established yet.
Project Heritage and Alternatives
Atomic Chat FAQ
Atomic Chat runs open-weight LLMs pulled from Hugging Face, including Llama, Qwen, Mistral, Gemma, and Phi, using its own llama.cpp fork, upstream llama.cpp, or the MLX-VLM engine on Apple Silicon.
Atomic Chat runs a local server at http://localhost:1337/v1 that mirrors the OpenAI API, so you can point the official OpenAI SDK — or any client built for it — at Atomic Chat just by changing the base URL.
Atomic Chat supports macOS 13.6+, Windows 10/11 x64, and Linux distributions with glibc 2.35 or newer, plus mobile apps for iOS and Android through the App Store and Google Play.
Atomic Chat's local server defaults to listening on 127.0.0.1 only, rather than your network, keeping it a private LLM chat setup — the README states conversations and API keys stay on your own machine unless you deliberately expose the server or add a cloud provider key.
Atomic Chat is released under the Apache 2.0 license, per its GitHub repository.
Atomic Chat's codebase traces back to an early fork of Jan, the open-source local-AI app from Menlo Research, per the README's heritage note. It has since built its own inference engines, speculative-decoding options, and agent integrations rather than tracking Jan's codebase.
The problem it solves
Running an open-weight model locally usually means picking one inference engine and living with its trade-offs — llama.cpp's broad CPU/GPU coverage, or MLX's Apple Silicon speed, but rarely both behind one API. Atomic Chat's README frames its answer as three separate engines (a TurboQuant llama.cpp fork, upstream llama.cpp, and MLX-VLM) exposed through a single OpenAI-compatible endpoint, so a coding agent or script written against localhost:1337/v1 keeps working no matter which backend is actually loaded underneath.
Who should try it — and who should skip
Try Atomic Chat if you already juggle local and cloud models and want one API surface plus a built-in agent launcher instead of stitching together separate tools. It's also a reasonable pick if you specifically want the TurboQuant KV-cache or MLX-VLM speedups on Apple Silicon. Skip it if you just want the lightest possible chat window — the engine-switching, speculative-decoding toggles, and multi-provider setup are more surface area than a single-model, single-backend app like a plain llama.cpp front end.
Related repositories
Want a second opinion on Atomic-Chat?
Ask an AI that can read this page — one click and you get its take on Atomic-Chat.