TopGit
GitHub Repo Review

XiaoZhi AI Chatbot: Voice AI on ESP32

78/xiaozhi-esp32
XTopGit review image for 78/xiaozhi-esp32
Review by Topgit.dev for 78/xiaozhi-esp32, with GitHub repository stats and README context.
Quick verdict

XiaoZhi AI Chatbot runs voice AI on ESP32 hardware, using MCP as its control plane for both IoT devices and cloud AI extensions. It is free under MIT and ships 171 pre-built firmware images, so trying it costs nothing beyond a supported board. Worth it if you want an embedded voice agent with a real protocol for extensibility; skip it if your stack doesn't include a cloud LLM, because the device itself only handles audio I/O and MCP messaging — the reasoning lives upstream.

Stars
★ 33.5k
Forks
⑂ 8.0k
Contributors
👥 165
Language
C++
License
MIT
Topic
AI Tools
Updated
Nov 2026
Homepage
GitHub

What is XiaoZhi AI Chatbot

XiaoZhi AI Chatbot is open-source ESP32 firmware that turns supported boards into voice AI agents. It uses Model Context Protocol (MCP) to communicate with cloud LLMs and expose local hardware simultaneously — so a query like 'turn off the bedroom light' can reach a smart home MCP server, while a conversation routes to Qwen or DeepSeek. Wake word detection and speaker recognition run on-device via ESP-SR; the cloud handles everything else. Binaries are pre-built for 171 board/chip combinations, and the project is MIT licensed for any use.

Core Features Implemented

  • ✓Two transports — WebSocket or MQTT+UDP hybrid — with automatic fallback, suitable for different network conditions and server setups.
  • ✓Offline voice wake-up via ESP-SR with customizable wake words; speaker recognition via 3D Speaker identifies who's talking without a cloud round-trip.
  • ✓Full-duplex voice pipeline: ASR → LLM → TTS with Opus audio streaming. Hardware with AEC support enables realtime conversation without pauses between send and receive.
  • ✓Device-side MCP exposes Speaker, LED, Servo, GPIO, and more to the cloud LLM. Cloud-side MCP extends the AI with smart home control, desktop operation, search, and email.
  • ✓Pre-built firmware for 138 board ports (ESP32, C3, C5, C6, S3, P4) connects to xiaozhi.me by default; self-hosted server options in Python, Java, and Go are community-maintained.
  • ✓OLED/LCD display output with emoji and rich expressions; optional camera input on supported boards for vision-capable interactions.
How this repository's GitHub stars have grown over time. Source: star-history.com.View the star history ↗

Why Voice AI on Microcontrollers

Putting a useful AI assistant on embedded hardware requires more than a speech-to-text model. You need a reliable audio pipeline, a way to extend the AI's capabilities to real-world devices (lights, servos, GPIOs), and a communication channel that survives spotty Wi-Fi — all without burning through expensive cloud bandwidth on every wake word. Generic ESP32 audio projects solve the first problem and ignore the other two. XiaoZhi addresses all three by routing audio through a cloud LLM while using MCP to expose device control both directions — cloud to device and device to cloud.

Flashing Firmware Without a Dev Environment

If you want to flash without a toolchain, download a pre-built binary from the releases page — the README links to a Feishu tutorial walkthrough for several popular boards (M5Stack, ESP32-S3-BOX-3, and generic ESP32-S3). Connect the board, flash the firmware, and it will connect to the official xiaozhi.me server where you can register and use Qwen's realtime model at no cost. For development, install ESP-IDF v6.1 (v6.0.1 is the minimum, v5.x is unsupported), the ESP-IDF VSCode/Cursor plugin, and clone the repo — Linux is recommended over Windows for faster builds and fewer driver issues.

Supported Boards and Use Cases

  • •Beginner voice assistant — flash a binary to an ESP32-S3-BOX-3 or M5Stack CoreS3, register at xiaozhi.me, and talk to Qwen in minutes without writing any code.
  • •IoT voice controller — MCP device-side lets the LLM toggle lights, control servos, or read GPIO state, bridging natural language to physical hardware without a separate hub.
  • •Embedded AI robot — combine speaker recognition, offline wake-up, and camera input on ESP32-P4 for a robot that identifies people and responds contextually.
  • •Smart home MCP extension — point xiaozhi at a cloud MCP server that wraps your smart home API, letting any LLM control your actual devices through conversation.
  • •Voice-enabled product prototype — MIT license means commercial use is unrestricted; ship the firmware as-is or build a custom release for your own board.

Strengths

  • ✓MCP protocol integration on both device side (Speaker, LED, Servo, GPIO control) and cloud side (smart home, PC operation, knowledge search) — bridges local hardware with extended LLM capabilities.
  • ✓Extensive board matrix: 138 board directories and 171 release variants across ESP32, ESP32-C3/C5/C6, ESP32-S3, and ESP32-P4 platforms, with pre-built firmware for beginners who don't want to set up ESP-IDF.
  • ✓Multi-modal on supported hardware: voice with offline wake-up via ESP-SR (customizable wake words), Opus streaming, speaker recognition via 3D Speaker, camera vision input, and OLED/LCD displays with emoji support.
  • ✓Dual transport options — WebSocket and MQTT + UDP hybrid — let developers trade off latency versus reliability depending on the use case.
  • ✓MIT licensed with a free tier on xiaozhi.me offering Qwen real-time model access, plus server implementations in Python, Java, and Go for self-hosting.

Known Limitations

  • △ESP-IDF v6.1 is now the minimum; ESP-IDF 5.x is unsupported, so older ESP32 variants are excluded outright.
  • △Compilation is noticeably slower on Windows and driver quirks are common; Linux is the documented preference for development.
  • △LLM inference runs on a cloud API (Qwen, DeepSeek), so ongoing costs apply — the firmware is free but the AI is not.
  • △Advanced audio features (full-duplex, realtime end-to-end) require AEC-capable hardware; not all supported boards have it.
  • △Camera and display features depend on specific board hardware — not a universal guarantee across all 138 board variants.

Alternative ESP32 AI Chatbot Projects

ESP32-S3-BOX-Lite — Espressif's own voice assistant board; ships with AT commands and a cloud service rather than open MCP firmware.ReSpeaker USB 6-Mic Array — higher-quality microphone array for Pi or x86, different firmware model, no built-in display or battery.AI-Thinker ESP32-S3-WIFI-LAMP — ESP32 with display and touch; supports voice but lacks the structured MCP IoT layer this project provides.

Frequently Asked Questions

Which ESP32 chips does xiaozhi esp32 support (ESP32-S3, C6, P4)?

XiaoZhi AI Chatbot supports ESP32, ESP32-C3, ESP32-C5, ESP32-C6, ESP32-S3, and ESP32-P4. ESP32-S3 and ESP32-P4 offer the best voice and camera capabilities, while ESP32-C variants are suited for lower-power builds.

Does xiaozhi esp32 work with DeepSeek or Qwen large language models?

Yes. The README states XiaoZhi AI Chatbot uses Qwen and DeepSeek as its primary cloud LLMs. The official xiaozhi.me server offers Qwen's realtime model free for personal accounts.

What communication protocol does xiaozhi esp32 use (MQTT, WebSocket, UDP)?

XiaoZhi AI Chatbot offers two transports: MQTT combined with UDP for hybrid message delivery, and WebSocket as an alternative. MQTT+UDP is the primary transport, with cryptographic code migrated to PSA Crypto in recent IDF 6 updates.

Can xiaozhi esp32 run without an internet connection (offline wake word)?

Partially. The wake word engine and speaker recognition run entirely on-device using ESP-SR — no internet needed to activate the device or identify a speaker. Full conversational AI still requires a cloud LLM round-trip.

How many board variants and releases are available for xiaozhi esp32?

The project maintains 138 board directories and 171 release variants covering popular hardware including M5Stack CoreS3, ESP32-S3-BOX-3, LILYGO T-Circle-S3, SenseCAP Watcher, and dozens more. Binaries are pre-built for most combinations.

Is xiaozhi esp32 free for commercial use under MIT license?

Yes. XiaoZhi AI Chatbot is released under the MIT license, which permits any use including commercial applications at no cost. The trade-off is that calling the cloud LLM API (Qwen, DeepSeek) has its own pricing separate from this project's license.

Who should try it — and who should skip

Pick this up if you want an embedded voice AI with a real protocol for extensibility and a low floor for trying it (171 binary builds exist). Hardware tinkerers building IoT products, educators demonstrating voice AI on real hardware, and developers who want MCP on a microcontroller will get the most out of it. Stay away if you need the full experience on a tight budget (LLM API costs), if your board isn't in the 138-variant matrix, or if you are on Windows and unwilling to deal with IDF compilation quirks.

Related repositories

Source & attribution

Repository: https://github.com/78/xiaozhi-esp32 — MIT licensed, C++ ESP-IDF project.

GitHub data · last synced Oct 20, 2026Reviewed by Henry
← Back to TopGit

Still deciding about xiaozhi-esp32?

One click hands the question to an AI along with this page — see what it says about xiaozhi-esp32.

GitHub