Echokit Server, developed by Second State, is an MCP server tool built with Rust, designed as an Open Source Voice Agent Platform. It functions as the central coordinator for voice agent systems, integrating various external AI services for speech recognition, language models, and text-to-speech. This platform supports advanced features like tool use capabilities through MCP server integration. It enables developers and builders to create and extend their own real-time voice AI systems, offering flexibility for both cloud-based and local deployments.

What does Echokit Server connect to?

Echokit Server connects to an EchoKit device, an ESP32-based hardware, and integrates with multiple external AI services. These include OpenAI-compatible API endpoints for speech recognition (ASR), large language models (LLM), and text-to-speech (TTS). It also supports additional MCP servers for extended AI functionality and tool use capabilities. Compatible services confirmed include Google Gemini Live services, Groq, ElevenLabs (streaming), Qwen Real-Time, local Whisper servers, Ollama, Piper Flask wrapper, and Silero VAD server.

Core features and tools

Echokit Server orchestrates a real-time ASR → LLM → TTS pipeline, acting as the central voice agent system coordinator. It integrates with external OpenAI-compatible services for all three stages. The server supports WebSocket as its primary transport for streaming voice. It extends functionality by integrating with other MCP servers using "http_streamable" and "http" types. Configuration is managed through a config.toml file, which includes sections for asr, llm, tts, and mcp_server integrations.

Who is it for?

Echokit Server is intended for developers, students, builders, and the open-source community. It caters to those interested in constructing and extending their own voice-AI systems or setting up AI labs. This includes individuals who wish to customize LLM endpoints, define LLM prompts, configure speech models, and integrate specialized MCP servers for voice agent functions. Skip this if you require a voice AI solution that does not involve extensive configuration and integration of external services.

How to get started

Authentication for Echokit Server primarily involves configuring API keys for the external AI services it integrates with. These API keys, such as for Groq and ElevenLabs, are specified within the config.toml file.

Use cases

• Real-time Voice AI Assistant: Powering an EchoKit device for natural, real-time conversations via an ASR → LLM → TTS pipeline.
• Custom AI Agent Development: Allowing developers to customize LLM endpoints, define LLM prompts, configure speech models, and integrate MCP servers.
• Local/Offline Voice AI Deployment: Enabling setup of a voice AI pipeline using local components like Ollama, local Whisper, and Piper.

Echokit Server from Second State offers a flexible foundation for real-time voice AI systems. It coordinates external ASR, LLM, and TTS services, supporting both cloud and local deployments. Its design allows for significant customization and integration with other MCP servers for specialized agent capabilities. Developers and builders can craft tailored voice assistants and AI labs. Try setting up a local voice AI pipeline using Ollama and local Whisper to experience its offline deployment potential.