Anything LLM from Mintplex Labs is a full-stack AI application designed for the mcp platform, connecting to various Large Language Models and embedding models. Built primarily with JavaScript/TypeScript and a Node.js backend, it offers both desktop and server/self-hosted deployment options. This free program focuses on data privacy and control, enabling users to interact with their documents and automate tasks using AI agents. Its flexible architecture supports both local and cloud-based AI providers, making it suitable for diverse operational environments.

What does Anything LLM connect to?

Anything LLM connects to diverse LLMs and embedding models, including local options like Ollama and LM Studio, alongside cloud services such as OpenAI and Google Gemini Pro. It uses SQLite for application data and LanceDB for embeddings, with support for other vector databases like Pinecone. External services are integrated for document ingestion via web scraping, and it can transcribe audio using Whisper. This setup allows for flexible data management and interaction with various AI capabilities.

Core features and tools

Anything LLM exposes several callable tools. The Developer API / REST API offers programmatic interaction with endpoints like "/api/v1/workspace/new" and "/api/v1/users/{id}/issue-auth-token", with interactive documentation at "/api/docs". Agent Skills/Tools enable built-in agents to browse the web, execute code, manage files, and call external APIs, including an Authenticated Scraping tool. A AnythingLLM CLI provides terminal interaction. Additionally, AnythingLLM Embed is a widget for embedding chat functionality into websites.

Who is it for?

Anything LLM serves individuals and teams seeking a private, self-hostable AI solution that prioritizes data privacy and control. It also benefits developers and organizations needing to integrate local LLMs into custom projects or manage internal AI knowledge bases. It supports interaction with proprietary documents and automates multi-step tasks. Skip this if direct Okta-based authentication is a native requirement for your multi-user setup.

How to get started

Access is granted through a standard API key (Bearer token) generated via the UI. These keys are included in the Authorization header of HTTP requests.

Use cases

• Document Chat and Retrieval-Augmented Generation (RAG): Upload diverse document types (e.g., PDFs, codebases, audio) into isolated workspaces for querying and receiving answers grounded in specific content, useful for tasks like legal contract analysis.
• AI Agent Automation: Utilize built-in or custom AI agents to automate complex, multi-step tasks such as web searching, authenticated data scraping, executing code, or making calls to external APIs.
• Local Meeting Transcription and Summarization: Automatically transcribe and summarize meetings on a local machine, generating action items, key decisions, and full transcripts without requiring external cloud services.

Anything LLM provides a self-hostable AI application for managing and interacting with private data across various LLM providers. Its multi-user capabilities, agent automation, and document-grounded chat features make it suitable for teams and developers focused on data privacy. The program supports diverse deployment methods, from desktop installers to Docker and Kubernetes.