Route local AI assistant traffic through a centralized, self-hosted gateway that applies real-time call compression and context optimization to reduce inference spend. Anyray Connect, from Othentic Labs Ltd, ships as a signed Windows executable for engineering teams and DevOps. It exposes an OpenAI-compatible API, offers spend attribution by team and project, and includes a fail-open timeout so developer workflows keep running. The tool fits teams that run LLMs at scale and require on-premise governance.

What does Anyray Connect actually do for developer LLM traffic?

Anyray Connect routes requests from local AI tools through an in-house gateway, rewriting endpoints so assistants and SDKs use the optimizer without code changes. The client-side executable automatically configures compatible tools to point to the gateway, and the gateway applies deterministic, rule-based transforms for compression and context trimming. The setup targets environments where prompt content must remain inside company infrastructure, and it exposes an OpenAI-compatible API so existing integrations continue to work.

Does it affect workflows or require engineering changes?

The tool works as a drop-in replacement for existing LLM endpoints, so teams do not need to alter assistant code or SDK calls. Moreover, the client uses a fail-open mechanism with an 800–1200 millisecond timeout that forwards requests unchanged if the optimizer is slow, which preserves developer productivity. Configuration occurs at the client and gateway layer, and installers are distributed as a single signed Windows executable or via the Windows Package Manager.

Is it safe to operate inside a corporate environment?

Safety is designed into the self-hosted architecture; the gateway and optimizer run inside your infrastructure (for example, Docker or Kubernetes), and prompt/response content stays on premises. In addition, the optimizer uses deterministic transforms to keep outputs stable, and metering sends only content-free metadata for spend attribution. These design choices limit external exposure while providing per-team and per-project usage reporting.

A targeted tool for engineering teams that already run self-hosted infrastructure

Anyray Connect suits engineering and DevOps teams that need granular governance of LLM traffic and already operate containerized or on-premise services. It preserves existing integrations by exposing a standard API and keeps sensitive prompts inside the network. The main trade-off is the operational requirement: teams must host and maintain the gateway infrastructure to benefit from the optimizer and attribution features.