Xiaozhi Esp32 refers to the client firmware for ESP32 devices, enabling them to function as AI chatbots. The associated backend, Xiaozhi Esp32 Server, connects these devices and integrates with various Large Language Models, including Qwen, GPT, and OpenAI-compatible APIs. Developed by 78, this free, open-source platform acts as a central hub for managing, controlling, and communicating with connected ESP32 hardware. It functions as an AI gateway and intelligent voice robot platform, supporting diverse server implementations in Python, Java, and Go, with simplified Docker deployment options.

What does Xiaozhi Esp32 connect to?

The Xiaozhi Esp32 client firmware connects to the Xiaozhi Esp32 Server, which then integrates with multiple Large Language Models (LLMs) like Qwen, DeepSeek, GPT, and Ernie Bot. It also supports any OpenAI-compatible API providers, including Ollama, Dify, FastGPT, Coze, and Xinference. This server acts as an open-source backend service for managing, controlling, and communicating with ESP32 devices, serving as a command center, relay hub, and AI gateway for connected hardware, also functioning as an intelligent voice robot platform.

Core features and tools

The server exposes callable tools for device registration and command routing. It supports plugin modules to add new functions such as weather retrieval, music playback, or smart home control. Documented callable actions include controlling devices with commands like "turn on the LED on my ESP32" and querying states such as "What lights are on?". For the Java implementation, examples include List Resources, Get Details, and Create New. It also exposes WebSocket endpoints for real-time interaction and supports MQTT and UDP protocols.

Who is it for?

Xiaozhi Esp32 Server is for developers building IoT applications with ESP32 devices, hobbyists, and professionals creating intelligent IoT systems. It suits makers and engineers interested in open, hackable AI voice assistants, and those involved in AI hardware development and prototyping. This platform supports creating voice-controlled devices and multi-terminal AI chatbots. Skip this if you are not working with ESP32 hardware or integrating with Large Language Models.

How to get started

The server handles authentication and command routing for connected devices. Device setup requires connecting the ESP32 client to the xiaozhi.me platform by logging in and adding a new device using a 6-digit verification code. The ESP32 client needs configuration of WiFi network information and the MCP server endpoint URL.

Use cases

• Voice-controlled IoT devices: An ESP32 device receives a voice command, sends it to the MCP server, which then invokes a registered tool on the device to execute the action.
• AI Chatbot with multi-terminal control: An ESP32 device acts as a voice interaction entry, utilizing cloud-based LLMs via the MCP protocol for controlling various terminals.
• Smart home integration: The server integrates with smart home systems like Home Assistant for unified control of home devices through the MCP interface and large language model calls.

Xiaozhi Esp32 provides a flexible backend for connecting ESP32 devices to AI models, enabling voice control and smart automation. Its open-source nature and multi-language server implementations offer adaptability for various IoT projects. The platform supports real-time communication and custom plugin modules to extend functionality.