AI Gateway by Ferro Labs is a free, mcp category program built with Go, distributed as a single static binary. It acts as a control plane, connecting AI clients and agents to various Large Language Model providers and Model Context Protocol servers. This gateway offers a unified OpenAI-compatible API endpoint, simplifying the interaction with diverse AI backend services. It supports streamable HTTP transport, ensuring efficient handling of stateful sessions and multi-part JSON-RPC messaging for production AI deployments.

What does AI Gateway connect to?

The AI Gateway connects to multiple Large Language Model providers, including OpenAI, Anthropic, Bedrock, and Azure. It also integrates with various Model Context Protocol servers. Functioning as an AI Gateway or MCP Gateway, it serves as a central control plane. This position allows it to sit between AI clients or agents and these diverse backend services, managing their interactions and requests effectively.

Core features and tools

The AI Gateway exposes a single OpenAI-compatible API endpoint for routing LLM requests. It orchestrates tool calls by injecting tool definitions and driving the agentic tool-call loop to backend MCP servers. It does not expose specific named callable tools of the gateway itself for AI agents to directly invoke. Instead, it provides a unified interface through which agents can access tools exposed by upstream LLM providers and MCP servers. It supports streamable HTTP transport and is compatible with the standard OpenAI SDK, NullClaw, AgentZero, OpenClaw, Claude Code, GPT, Gemini, and Open Interpreter.

Who is it for?

AI Gateway is designed for teams deploying AI in production, individual developers, and enterprises. It assists with managing interactions between AI clients and various LLM providers and MCP servers. Its capabilities support complex agentic workflows and provide observability for AI operations. Skip this if your primary need is a standalone AI agent with built-in, direct-callable tools from the gateway itself.

How to get started

The server authenticates using an API key passed as an environment variable. For AI agents and services, access is granted through a standard OAuth 2.1 authorization flow. Setup involves setting the OPENAI_API_KEY and MASTER_KEY environment variables.

Use cases

• Routing LLM requests across multiple providers with features like caching, guardrails, A/B testing, and usage controls.
• Orchestrating MCP tool calls, enabling AI agents to autonomously discover, authenticate, and execute multi-step workflows by interacting with backend MCP servers.
• Providing observability through OpenTelemetry tracing and Prometheus metrics, along with an operations console for monitoring traffic, financial usage, provider health, routing, plugins, request logs, and audit trails.

AI Gateway by Ferro Labs offers a centralized control plane for managing AI client interactions with LLM providers and MCP servers. Its design supports complex agentic workflows and provides essential operational visibility. The program is distributed freely and built with Go, ensuring a single static binary for deployment. It integrates with existing identity providers for authentication and offers multiple distribution methods.