Bubbaloop, developed by Kornia, functions as an MCP (Model Context Protocol) server and a VLM (Vision Language Model) Serving Platform for Multimodal AI. Built with Rust, its core daemon and MCP server are distributed as a single binary. It serves as the sole control interface for various hardware AI agents, including cameras, sensors, robots, and IoT fleets. Bubbaloop empowers users to interact with and automate physical systems using natural language, making advanced AI capabilities accessible for real-world applications.

What does Bubbaloop connect to?

Bubbaloop connects to and manages a variety of hardware AI agents. This includes devices such as cameras, sensors, robots, and entire IoT fleets. Its primary role is an MCP (Model Context Protocol) server, acting as the sole control interface for these AI agents. Additionally, Bubbaloop operates as a VLM (Vision Language Model) Serving Platform for Multimodal AI. The daemon integrates an agent runtime, which functions as a multi-agent Zenoh gateway, alongside the MCP server, utilizing Zenoh for efficient pub/sub messaging.

Core features and tools

Bubbaloop exposes 47 MCP tools for interaction, complemented by several agent-internal tools. The daemon-side agent-internal tools, which are not exposed via MCP, include grab_frame, memory_search, memory_forget, schedule_task, create_proposal, read_file, write_file, run_command, and publish_to_topic. The MCP server automatically starts on port 8088. The system supports compatible AI clients such as Gemini (recommended), Claude, and Ollama for local use, facilitating diverse AI interactions and capabilities.

Who is it for?

Bubbaloop is designed for users who seek to interact with and automate physical systems using natural language. This includes individuals managing cameras, sensors, robots, and IoT fleets. It enables advanced control and automation for hardware devices through AI. Specific applications involve querying connected sensors or orchestrating federated IoT and robotics fleets. Skip this if your workflow does not involve controlling physical hardware AI agents with natural language.

How to get started

The server authenticates using an API key passed as an environment variable or through a login command. The dashboard's chat interface communicates via HTTP+SSE, and the CLI uses a REST API.

Use cases

• Natural language interaction with hardware, for example, querying connected sensors with "bubbaloop agent chat 'What sensors do I have?'".
• Automating physical systems and managing federated IoT/robotics fleets.
• Performing vision language model scene descriptions on camera frames, utilizing the "grab_frame" tool to fetch a JPEG for AI analysis.

Bubbaloop offers a powerful platform for controlling diverse hardware AI agents and serving multimodal AI. It simplifies interaction with physical systems through natural language and supports leading AI clients. Kornia's Bubbaloop provides tools for automation, sensor querying, and VLM scene descriptions. This Rust-built system unifies various AI capabilities for practical hardware management. Try interacting with your connected sensors by using "bubbaloop agent chat 'What sensors do I have?'".