78. OmniRoute

The lesson

OmniRoute (omniroute.online) is a free, open-source AI gateway released under the MIT licence. Its slogan is "Never Stop Coding": it sits between your coding tools and the growing ecosystem of LLM providers, so a request always finds a working model instead of stopping at a quota or rate-limit wall. It presents one local OpenAI-compatible endpoint and routes across roughly 250-268 providers, with more than 90 offering a free tier and 11 free forever (for example Kiro, Pollinations, and LongCat). No credit card is needed to start.

The core idea is a single endpoint. You install OmniRoute with one command (npm install -g omniroute, a Docker image, or the Electron desktop app for Windows, macOS, and Linux), then launch it with `omniroute`. The API and the dashboard come up together on port 20128. You connect a provider in the dashboard, create an API key, and point every tool at http://localhost:20128/v1. Because OmniRoute translates between the OpenAI, Claude, Gemini, and Responses API formats, any tool that can set a base URL and an API key works: Claude Code, Codex, Cursor, Cline, GitHub Copilot, Gemini CLI, OpenCode, Kilo Code, Continue, Roo Code, Antigravity, and Hermes Agent all point at the same endpoint.

Automatic fallback is the headline feature. If one provider runs out of quota or rate-limits you mid-task, OmniRoute switches to the next provider in milliseconds with no downtime. Routing is controlled by 17-18 strategies grouped by goal: drain a paid subscription first, spread load across keys, always pick the cheapest model, keep long contexts on models that fit, randomise to avoid fingerprinting, or let the smart `auto` mode score nine factors and pick for you. You can also use named combos such as auto/coding, auto/fast, auto/cheap, auto/offline, and auto/smart.

Resilience is layered three deep so one failure never takes everything down: a circuit breaker per provider, a cooldown per connection (a bad key), and a lockout per model. Other keys and models keep serving while the broken piece recovers lazily, with no background timers. This goes beyond the flat three-tier fallback most other routers ship.

Cost and token savings are a major focus. OmniRoute tracks every free tier across the providers and drains the quota you would otherwise lose, honestly de-duplicated across shared accounts, for roughly 1.4 billion free tokens a month. On top of that, a stacked compression pipeline (Session-Dedup, CCR, RTK, Headroom, Relevance, Caveman, and LLMLingua-2) cuts 15-95% of eligible tokens from tool output and context, which matters on long, tool-heavy coding sessions. Output styles let you ask for terse prose or less generated code.

The provider catalogue is the most complete of any open router. It spans OAuth providers you sign into once (Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Kiro, Qoder, Windsurf, Zed, and more), API-key providers (OpenAI, Groq, NVIDIA, Cerebras, Mistral), web-cookie providers that wrap a provider's own web app, local providers (Ollama, LM Studio, vLLM), free-forever tiers, web-search, audio (TTS/STT such as ElevenLabs and Deepgram), image, video, embeddings, rerank, upstream proxies, and cloud agents (Codex Cloud, Devin, Google Jules). The full auto-generated list lives in docs/reference/PROVIDER_REFERENCE.md on the GitHub repo and is regenerated from the provider constants file.

Production-grade extras round it out: persistent hybrid memory (FTS5 keyword plus Qdrant vector recall), a built-in MCP server that exposes the gateway itself as about a hundred tools, an A2A (agent-to-agent) JSON-RPC server, prompt-injection guardrails with opt-in PII redaction, an evaluation framework, semantic caching, TLS fingerprint stealth, a three-level proxy, a Next.js dashboard with 42 locales, analytics, and gamified savings tracking. The project is built in the open with tens of thousands of GitHub stars, hundreds of contributors, and more than 25,000 automated tests.

How this relates to TPEE: TPEE itself never talks to OmniRoute. The hard offline rule means tpee.exe makes zero network calls and runs no MCP server. The safe pattern is to design and score a prompt inside TPEE, then copy it manually into a separate tool (for example Hermes Agent configured with OmniRoute as its provider, or Claude Code pointed at localhost:20128/v1) and let that tool use the gateway. You can also use TPEE simply to document which OmniRoute combo or model a project intends to use. Quick reference: install with `npm i -g omniroute`, start with `omniroute`, open the dashboard at http://localhost:20128/dashboard, list models with `curl localhost:20128/v1/models`, and set every tool's base URL to http://localhost:20128/v1 with a dashboard-generated key.

Check yourself