Lemonade: Local AI Server with GPU and NPU Acceleration
/lemonade
Lemonade delivers on the promise of free, private AI inference on consumer hardware — it works, it covers text and multi-modal generation, and it speaks the OpenAI API your existing stack already understands. The catch is that performance depends entirely on your GPU or NPU, and several backends (NPU LLM, ROCm VLLM, TheRock audio) are marked experimental, so the smoothest experience is still on CUDA or Vulkan with an NVIDIA card. If you have the hardware and want zero per-token costs with full data control, it is worth the setup friction.
What is Lemonade?
Lemonade is a local AI server that runs LLMs, image generation, speech synthesis, and transcription entirely on your own GPU or NPU. It provides standard OpenAI-compatible API endpoints so existing apps and code connect without modification, while the CLI handles model discovery and execution. Built in C++ with contributions from AMD engineers, it targets consumers and developers who want cloud-level AI capability with zero per-token cost and full data privacy.
Key Capabilities for Local AI
- ✓OpenAI-compatible API at localhost:13305 — existing code using openai-python, openai-node, and other standard clients works without changes
- ✓Multi-modal inference covering text generation (GGUF, FLM, ONNX), image generation (SDXL-Turbo, Trellis), speech synthesis (Kokoro), and transcription (Whisper)
- ✓Hardware acceleration across CUDA, Vulkan, ROCm, Metal, and XDNA2 NPU backends — the same model can run on NVIDIA, AMD, or Apple Silicon depending on what is installed
- ✓AMD Ryzen AI and Radeon optimization with dedicated backends for XDNA2 NPUs and Strix Halo iGPUs, built with IRON/MLIR-AIE
- ✓Embeddable Lemonade: a portable binary you can bundle into your own application so end users get local AI without installing a separate server
- ✓Model aliasing for environment-independent naming and instant active-standby failover between model targets
- ✓Hybrid cloud offload: route to OpenAI, Fireworks, or OpenRouter alongside local models (experimental)
- ✓Mobile companion apps for iOS and Android that connect to a running Lemonade instance
Getting Started with Lemonade
Download the MSI installer for Windows from the releases page, use the standard package manager on Linux, or grab the macOS build. Docker and source build instructions are also available. After installation, run `lemonade backends` to see which inference engines and hardware acceleration paths are available on your system, then `lemonade list` to browse available models and `lemonade pull <model>` to download one. The server starts automatically and listens on localhost:13305. Mobile apps for iOS and Android connect to a running instance.
Running Models and Connecting Apps
The CLI is the primary interface. `lemonade run <model>` starts a model and opens a chat prompt. `lemonade list` shows which models are available locally, `lemonade pull <name>` downloads from Hugging Face or ModelScope, and `lemonade backends` reports what your hardware supports. For application integration, configure any OpenAI-compatible client to point to `http://localhost:13305/api/v1` with any non-empty string as the API key — no real key required. Python, Node.js, Go, Java, C++, C#, Ruby, Rust, and PHP client examples are linked in the README. The server also supports Anthropic and Ollama API-compatible endpoints.
Integrating Local AI into Your Applications
- •Replace cloud AI API calls in a production application with a local Lemonade server, eliminating per-token costs while keeping the same OpenAI client library
- •Bundle Embeddable Lemonade into a desktop or mobile app to give end users on-device AI that auto-optimizes for their specific GPU or NPU
- •Run a privacy-sensitive LLM pipeline — legal, medical, or financial documents processed locally, never sent to a cloud provider
- •Power a multi-modal creative workflow: generate an image with SDXL-Turbo, describe it with a vision-capable LLM, and synthesize a voiceover with Kokoro, all on the same machine
- •Develop and test AI features locally before deploying to cloud APIs, using the same API interface for both environments
- •Use model aliasing to swap between a small fast model and a large capable model at runtime without changing application code
Strengths
- ✓Apache 2.0 license means free for personal and commercial use with no telemetry or licensing strings attached
- ✓OpenAI API compatibility covers a vast ecosystem of existing tools, libraries, and applications without requiring code changes
- ✓Genuine multi-modal stack — text, image, audio, and 3D generation in one server, runnable entirely offline
- ✓AMD hardware optimizations give Ryzen AI NPU and Radeon owners a meaningful alternative to NVIDIA for on-device inference
- ✓Embeddable Lemonade lets developers ship local AI inside their own apps without requiring users to install a separate server or understand inference configuration
- ✓Model aliasing and hybrid cloud offload provide deployment flexibility for teams moving between local and cloud inference
Hardware and Experimental Features
- △Performance is entirely hardware-dependent — running a 7B model on a CPU or a weak iGPU will be slow enough to be unusable for interactive applications
- △Several backends are marked experimental: llama.cpp-hrx for AMD gfx1100/gfx1151 GPUs, vllm for Strix Halo on Linux, TheRock audio generation on ROCm, and ds4 for Strix Halo
- △NPU-only LLM inference requires Windows and specific AMD XDNA2 hardware (Ryzen AI MAX+ Pro 395 or equivalent) — setup is not as plug-and-play as CUDA
- △The NPU LLM pipeline (ryzenai-llm) is Windows-only at this stage; Linux users have llama.cpp-hrx as an experimental NPU alternative
- △Most Linux and macOS installation paths lack the one-click installer simplicity of the Windows MSI, making the initial setup more involved for non-Windows users
Comparing Lemonade to Other AI Servers
Frequently Asked Questions
Yes. Lemonade is licensed under Apache 2.0 and imposes no usage fees or subscription costs. Because inference runs entirely on your local hardware, no prompts, responses, or user data are transmitted to any external server — only the model weight downloads from Hugging Face or ModelScope during setup touch the internet.
Lemonade runs text generation models in GGUF, FLM, and ONNX formats; image generation with Stable Diffusion and Trellis; speech synthesis with Kokoro; and transcription with Whisper and MoonShine. This covers the full pipeline from prompt to generated output across text, audio, image, and 3D modalities.
It targets NVIDIA GPUs from the Turing generation onward (CUDA), AMD GPUs through the ROCm stack, Apple Silicon (Metal), and AMD XDNA2 NPUs on Ryzen AI hardware. Vulkan and CPU-only backends are available for machines without a discrete GPU. Run `lemonade backends` after installation to see exactly which backends your system can use.
Yes. Lemonade exposes OpenAI-compatible API endpoints at localhost:13305, so any client library built for the OpenAI API — openai-python, openai-node, Go, Java, Ruby, and others — connects by changing the base URL and using a placeholder API key. It also supports Anthropic and Ollama API-compatible endpoints.
Embeddable Lemonade is a portable binary version of the runtime that developers can bundle directly into their own application. End users get on-device AI without installing a separate server, and the binary auto-optimizes for the hardware it detects — no branding, no separate installer, no telemetry from the Lemonade side.
Lemonade is licensed under Apache 2.0, which permits use, modification, and commercial redistribution with no royalty or attribution requirements beyond the standard Apache notice. The project also respects the licenses of the upstream libraries it depends on, including llama.cpp, whisper.cpp, Stable Diffusion, and Kokoro.
The problem it solves
Cloud AI APIs charge per token and route your prompts through third-party servers, creating ongoing costs and privacy exposure. Setting up local inference from scratch means piecing together inference engines, hardware acceleration layers, model downloads, and API servers — a fragmented experience that casual users and app developers cannot be expected to assemble.
Who should try it — and who should skip
Pick up Lemonade if you have an AMD or NVIDIA GPU and want to run LLMs or multi-modal AI in an application without paying per-token API fees or sending data to the cloud — particularly strong if you are on AMD Ryzen AI or Radeon hardware. Developers who want to ship on-device AI inside their own app should look closely at Embeddable Lemonade. Skip it if you only have an older CPU-only machine where inference will be impractically slow, or if you need the GUI-first experience of a dedicated chat interface rather than a CLI and API server.
Related repositories
Want a second opinion on lemonade?
Ask an AI that can read this page — one click and you get its take on lemonade.