Repository avatar
AI Tools
v0.7.0
active

kaeso

app.kaeso/kaeso

Manage a self-hosted Kaeso AI gateway: projects, API keys, models, routes, limits and usage.

Documentation


Kaeso

The open-source AI gateway you run yourself

One OpenAI-, Anthropic-, Gemini- and ElevenLabs-compatible API in front of every provider and your own model servers, for chat, embeddings, reranking, decisions, images, video, speech, transcription and music. Single sign-on, SCIM, roles, budgets, guardrails, alias routes, key pools, request logs, a playground and an MCP portal come with it, under the MIT license, with no paid edition.

CI Latest release OpenSSF Scorecard OpenSSF Best Practices License: MIT

Website · Documentation (Deutsch, 简体中文) · Live demo · Articles · Changelog


The Kaeso console with sample data: a request in the request log opens its inspector, which shows the app's OpenAI Chat Completions request sent to Anthropic Messages through the support alias; then the support alias route with a condition, a split by caller and fallbacks; the project's spend budget and rate limits; and the project's analytics

Install

On a Linux server or your computer with Docker and Compose 2.24 or newer:

curl -fsSL https://kaeso.app/install.sh | sh

It installs Kaeso with PostgreSQL, ClickHouse and Valkey into ./kaeso, starts it at https://localhost and prints the link that creates the first user. Add -s -- --domain kaeso.example.com to serve your own domain. The quick start continues from there, and install by hand sets up the same Compose package step by step.

On Kubernetes, each release publishes a Helm chart next to its images (Kubernetes):

helm install kaeso oci://ghcr.io/logicplanes/charts/kaeso \
  --namespace kaeso --create-namespace --values values.yaml --wait

On a cloud or a hosting platform, the platform templates run the same release:

Deploy to Render Deploy on Railway Deploy to DigitalOcean
Launch on AWS Deploy to Azure Open in Google Cloud Shell

Render · Railway · DigitalOcean · Fly.io · AWS · Azure · Google Cloud · Coolify · Dokploy · CapRover · Kubernetes

Updates install from the console's Updates page or with the installer's --update, and check the install's settings before they switch anything (upgrades).

Send your first request

Point the OpenAI SDK at Kaeso and call any model in your project, Claude included:

from openai import OpenAI
client = OpenAI(base_url="https://kaeso.example.com/v1", api_key="sk-kso-…")
print(client.responses.create(model="claude-haiku-5-5", input="Hello").output_text)

The Anthropic, Google Gen AI, ElevenLabs and Cohere SDKs work the same way with their base URL pointed at Kaeso. Kaeso translates the request and the reply between the formats (calling models).

Built in are Amazon Bedrock, Anthropic, AssemblyAI, Azure, Baidu Qianfan, Baseten, Cartesia, Cerebras, Cloudflare AI Gateway, Cohere, Deepgram, DeepInfra, DeepSeek, ElevenLabs, fal, Fireworks AI, Gemini Enterprise Agent Platform, Google AI Studio, Groq, Hugging Face, IONOS AI Model Hub, Jina AI, MiniMax, Mistral, Moonshot, Nebius Token Factory, Novita AI, NVIDIA, Ollama Cloud, OpenAI, OpenRouter, OVHcloud AI Endpoints, Perplexity, QwenCloud, Replicate, SambaNova, Scaleway, SiliconFlow, SpaceX (xAI), Standard Compute, StepFun, Tencent TokenHub, Together AI, TypeSafe, Upstage, Venice, Vercel, Volcengine Ark, Voyage AI and Z.ai (supported providers). A server of your own, such as vLLM, Ollama, LM Studio or llama.cpp, joins as a custom provider.

What you get

One format in, any provider out
Apps speak OpenAI Chat Completions or Responses, Anthropic Messages, Gemini or ElevenLabs; Kaeso encodes each request for the provider and translates the reply and the stream back (format translation).
More than chat
Embeddings, reranking in Cohere's format, decision models that answer typed questions with probabilities, images, video, speech, transcription, sound effects, music and OpenAI-format batches, with the same keys, limits and logs (API reference).
Routes that hold up
Alias routes with conditions, weighted splits and fallbacks, retries after dropped connections and 5xx answers, key pools that spread a project over several keys of one provider, and a response cache (alias routes).
Model lifecycle
Replies from retiring models carry Deprecation and Sunset headers, projects see who still calls them, aliases move to the replacement, and model lines such as claude-opus-latest follow the newest model (model lifecycle).
People and access
Single sign-on per organization over OpenID Connect for any number of people, roles and projects from your identity provider's groups, SCIM 2.0, organization and project roles with single rights, and audit logs (members and roles).
Spend under control
Hard or soft budgets and rate limits per project, member, service account or key, checked before the provider is called; prices from KPMR, shown in your currency; alerts by email or signed webhook (limits and budgets).
Guardrails
Built-in rules find keys, tokens, email addresses, IBANs and card numbers and mask or block them, and your own HTTP services can allow, block or rewrite a request before the model sees it (guardrails).
Logs you decide on
Every request with its model, status, tokens, cost, timing and caller; content only where the organization, the project or the request allows it; OpenTelemetry traces and Prometheus metrics for your own tools (request logs).
An MCP portal per project
Connect your MCP servers once, review their tools, and let claude.ai, ChatGPT, Cursor or any MCP client call them through one signed-in portal (MCP portal).
Verifiable releases
Every image, chart and release file carries signed build provenance that gh attestation verify checks, and the updater installs nothing whose signatures don't match (upgrades).

New in 0.7

Decision models, reranking, guardrails, webhooks, SCIM, key pools, batches and the Helm chart arrived in 0.7, with new providers from Amazon Bedrock and Azure to Deepgram and fal. The release article walks through them, and the changelog lists every change.

Two decision models side by side in the Playground answer a refund request: refund yes at 94 and 90 percent, the shipping team at 81 and 74 percent, and an urgency score

Decision models

A guardrail's built-in rules tried on sample text: an order number, an email address and an API key are replaced by placeholders before the model sees them

Guardrails

An OpenAI key assigned to a project with a weight of 30, beside the project's other OpenAI keys

Key pools

A deprecated model with its announcement, shutdown date, replacement and new prices, the aliases that still use it and who called it in the last 30 days

Model lifecycle

A webhook for the on-call team with its URL, format and events, and its last deliveries of budget alerts, limits reached and refused provider keys

Webhooks

An organization's provisioning settings with its SCIM base URL, token and last activity, above the mapping of the identity provider's groups

SCIM 2.0 and SSO groups

The console

The pages below show the console with sample data. The live demo is a read-only install to try it yourself.

Analytics: requests, tokens, cost and error rate for the last 30 days against the 30 days before, and a chart of requests per day

Analytics
Requests, tokens, cost and errors per organization, project, model, key and member, against the period before.

Request logs: every request with its time, status, model, duration, tokens with the cached share, and cost

Request logs
Every request with its model, status, tokens, cost, timing, route and caller, filtered by any of them.

The project's model catalogue: Anthropic and OpenAI models with their aliases, capabilities, context windows, prices and release dates, two of them marked deprecated

Models
A catalogue per project with the prices, capabilities and status of every model from KPMR, the Kaeso Provider & Model Registry.

An alias route in the editor: a condition sends enterprise requests to Claude Sonnet 5.5 with a fallback to GPT-6 Sol, and splits the rest by caller between Claude Haiku 5.5 and GPT-5.4 mini

Alias routes
Name the models behind an alias, with fallbacks, retries, conditions on headers and tokens, and weighted splits.

Limits: a hard monthly spend budget and a request rate for the project, and a soft daily token budget for a service account

Limits and budgets
Hard or soft spend, token and rate limits per project, member, service account or key.

Logging settings: whether request content and media are kept, and how many days metadata and content are kept

Logging
Whether request content and media are kept at all, and for how long, per organization and project.

How it works

flowchart LR
  accTitle: How Kaeso sits between apps and providers
  accDescr: Apps, SDKs and coding tools call the gateway in the OpenAI, Anthropic, Gemini or ElevenLabs format, and MCP clients call its MCP portal. The gateway resolves the model, checks rights, limits and guardrails, calls the provider in its own format, and writes usage and logs to PostgreSQL, ClickHouse and Valkey, which the console reads.
  apps[Apps, SDKs and coding tools]:::outside
  mcp[MCP clients]:::outside
  subgraph install [Your Kaeso install]
    direction LR
    gateway[Gateway]:::accent
    console[Console]
    data[(PostgreSQL, ClickHouse, Valkey)]
    gateway --- data
    console --- gateway
  end
  guardrails[Your guardrail services]:::outside
  providers[Model providers]:::outside
  servers[Your MCP servers]:::outside
  apps -- OpenAI, Anthropic, Gemini or ElevenLabs format --> gateway
  mcp -- MCP --> gateway
  gateway -- signed HTTP --> guardrails
  gateway -- the provider's format --> providers
  gateway -- MCP --> servers

The gateway is one Rust process with no garbage collector between your app and the provider. Run several behind Caddy or a load balancer and they share everything a request depends on (multiple gateways). The gateway benchmark in this repository measures the time and memory Kaeso adds, so you can run it on your own hardware.

How it compares

These tables compare Kaeso with the gateways people most often weigh it against, as their own documentation, pricing pages and licenses describe them. They were read on 21 November 2026 and every entry is sourced below. Products change: open an issue when an entry is out of date, and we'll correct it.

Yes means the free or open-source version has it. Paid means a paid plan or enterprise license has it. Partial is explained in a note. – means we didn't find it in their documentation.

Running it yourself

On your serversLicenseWritten inWeb consoleDocker ComposeHelm chart
KaesoYesMITRustYesYesYes
LiteLLMYesMIT, except enterprise/1Python2YesYesYes
BifrostYes3Apache 2.0GoYesYesYes
PortkeyYesMITTypeScriptLogs onlyYesPaid4
HeliconePartial5Apache 2.0TypeScriptYesYesYes
Kong AI GatewayYesApache 2.0, AI plugins paid6LuaYesYesYes
Agent Router (was Envoy AI Gateway)YesApache 2.0Go–YesYes
OpenRouterHosted only––Hosted––
Cloudflare AI GatewayHosted only––Hosted––

Teams and control

Single sign-onSCIMRolesAudit logBudgetsPII and secret rulesYour own guardrail service
KaesoYesYesYesYesYesYesYes
LiteLLMUp to 5 usersPaidPartial7PaidYesPartial8Yes
BifrostPaidPaidPaidPaidYesPaidPartial9
PortkeyPaidPaidPaidPaidPaidPaidYes
HeliconePaid–––Partial10––
Kong AI GatewayPaidPaidPaidPaidPaidPaidPaid
Agent Router––––Partial11–Partial12
OpenRouterPaidPaidPartial13PaidYes14Yes–
Cloudflare AI GatewayYesPaidPartial15YesYesPartial16–

APIs your apps can call

OpenAI Chat Completions and ResponsesAnthropic MessagesGeminiElevenLabsRerankOpenAI batches
KaesoYesYesYesYesYesPartial17
LiteLLMYesYesYes–YesYes
BifrostYesYesYesYesYesYes
PortkeyYesYesPartial18Partial19YesPaid
HeliconeYesPartial20Partial20–––
Kong AI GatewayPartial21Paid21Yes–Paid21Paid21
Agent RouterYesYes––Yes–
OpenRouterYesYes––YesYes
Cloudflare AI GatewayYesYesYes22Yes22Partial23–

Media

ImagesVideoSpeechTranscriptionMusic
KaesoYesYesYesYesYes
LiteLLMYesYesYesYesYes
BifrostYesYesYesYes–
PortkeyYesYesYesYes–
HeliconePartial24––––
Kong AI GatewayPaid21Paid21Paid21Paid21–
Agent RouterYes––Yes–
OpenRouterYesYesYesYes–
Cloudflare AI GatewayYesYesYesYes–

Routing and operations

Several keys per providerFallbacksRetriesModel retirementsMCP gatewayLogs without content
KaesoYes, weightedYesYesYes25YesYes
LiteLLMYesYesYesPartial26YesYes
BifrostYes, weightedYesYesPartial27YesYes
PortkeyYes, weightedYesYes–Partial28Paid
Helicone–YesYes––Yes
Kong AI Gateway–PaidPaid–PaidYes
Agent RouterPartial29YesYes–YesYes
OpenRouterPartial30YesYesPartial31–Yes
Cloudflare AI GatewayPartial32YesYes––33Yes

Paid plans, as published

ProductPaid plans
KaesoNone. Everything above is in the MIT-licensed release; there is no paid edition or support contract.
LiteLLMEnterprise license, priced on request.
BifrostEnterprise, custom pricing.
PortkeyPro at $49 a month and Enterprise at custom pricing; Portkey is now offered by Palo Alto Networks as Prisma AIRS AI Gateway.
HeliconePro at $79 and Team at $799 a month, Enterprise on request.
Kong AI GatewayKonnect Plus from $25 a month plus usage, with AI model proxies at $100 a month each; Enterprise at custom pricing.
Agent RouterNone; an Apache 2.0 project.
OpenRouterA 5.5% fee on credits on Standard and 8% on Business, Enterprise on request.
Cloudflare AI GatewayCore features free; a 5% fee on Unified Billing credits; logs billed as Workers Logs.

When another tool fits better

Kaeso runs as containers next to PostgreSQL, ClickHouse and Valkey, on at least 2 vCPUs and 4 GB of memory (requirements), and it has no hosted version and no paid support. If you want a hosted service with nothing to run, OpenRouter or Cloudflare AI Gateway fit better, and OpenRouter bills hundreds of models through one account. If your platform team already runs Kong or Envoy Gateway, Kong AI Gateway or Agent Router put model traffic under the same policies and Kubernetes resources. LiteLLM serves more kinds of endpoints, such as realtime, OCR and vector stores, and comes as a Python SDK as well as a proxy. Helicone goes further into observability, with sessions, evaluations, datasets and prompt management. If you need a vendor contract, LiteLLM, Bifrost, Portkey, Helicone and Kong sell enterprise plans.

TensorZero is not in the tables: its website says it is no longer maintained.

Notes

  1. The repository's license is MIT except for enterprise/ folders, whose code needs a BerriAI Enterprise license in production.
  2. GitHub lists Python as the repository's main language; its description reads "Rust core with Python SDK".
  3. Running several open-source Bifrost nodes against one PostgreSQL is not supported; clustering is part of Enterprise.
  4. The official chart pulls enterprise images with credentials Portkey shares.
  5. The self-hosted stack forwards OpenAI and Anthropic traffic; Vertex AI, Bedrock and Azure OpenAI are not supported there, and the AI gateway is documented as a cloud endpoint.
  6. Kong Gateway is Apache 2.0; most AI plugins, such as AI Proxy Advanced, AI Sanitizer and AI MCP Proxy, are part of AI Gateway Enterprise.
  7. Global proxy roles carry no enterprise label; roles scoped to organizations or teams are a premium feature.
  8. PII masking with Presidio is in the open-source version; secret detection and redaction are enterprise-only.
  9. No webhook guardrail in the open-source version; Go or WASM plugins can inspect requests, and named guardrail vendors are Enterprise.
  10. Cost-based rate limits per user or property, set with a request header.
  11. Token quotas per model, not money.
  12. Envoy's external authorization hook, not a guardrail feature of its own.
  13. Organization admins and members; workspace roles need a higher plan.
  14. The workspace budgets page says budgets are on every plan; the pricing page lists them as not included on Free.
  15. Cloudflare account roles and API token permissions; no role for AI Gateway members alone.
  16. Data loss prevention flags or blocks prompts and replies; two predefined profiles without a Zero Trust subscription, no masking documented.
  17. For providers whose batch API takes OpenAI's batches as they are, today OpenAI.
  18. Google Gen AI SDK requests pass through for Gemini models on Vertex AI only.
  19. Speech and transcription go through OpenAI-style /v1/audio endpoints; other ElevenLabs APIs through a custom host.
  20. Through a provider-specific proxy that Helicone maintains but no longer develops.
  21. Documented for AI Proxy from Kong Gateway 3.10 or 3.11; the open-source releases on GitHub are 3.9. The open-source AI Proxy takes Chat Completions.
  22. The provider's own API, passed through to that provider only.
  23. Cohere's API passes through to Cohere.
  24. One image model, through Google AI Studio.
  25. Deprecation, Sunset and Link headers, 410 after the shutdown, and aliases that move to the replacement.
  26. Deprecation reminders 30 days and 1 day before a provider's date, with a recommended replacement.
  27. Requests to a retired model are recognized and handled like other failures; no warnings or move to a replacement are documented.
  28. In Portkey's hosted and enterprise platform, not in the open-source gateway.
  29. Weighted backends, each with its own credentials.
  30. Several of your own keys, tried in the order you set; no weights.
  31. The models API returns a model endpoint's deprecation date.
  32. Several keys per provider, chosen per request by an alias header.
  33. MCP server portals are a separate Cloudflare One product.
Sources, all read on 21 November 2026

Kaeso: license, installation, Kubernetes, single sign-on and SCIM, roles, audit log, budgets, guardrails, model API, ElevenLabs API, rerank, batches, images, videos, audio, key pools, fallbacks and retries, model lifecycle, MCP portal, keeping content out of the log, requirements.

LiteLLM: license, enterprise license, repository, pricing, enterprise features, Docker, Helm, SSO, SCIM, roles, audit logs, budgets, PII masking, generic guardrail API, supported endpoints, Responses, Anthropic format, generateContent, pass-through, rerank, batches, images, speech, transcription, music, load balancing, fallbacks and retries, deprecation alerts, MCP gateway, logging.

Bifrost: license, repository, pricing, setup, multiple nodes, Helm, SSO and SCIM, RBAC, audit logs, budgets, guardrails, plugins, OpenAI, Anthropic, Google GenAI, ElevenLabs, Cohere, batches, media, video, keys, retries and fallbacks, MCP, content logging.

Portkey: license, repository, deployment, Helm chart, plans and features, SSO, audit logs, budgets, PII redaction, webhook guardrails, Responses, Messages, Vertex AI, ElevenLabs, rerank, batches, images, video, speech, transcription, load balancing, fallbacks, retries, MCP gateway, request logging, Prisma AIRS AI Gateway, where portkey.ai/pricing leads.

Helicone: license, repository, pricing, self-hosting with Docker, Kubernetes, custom rate limits, gateway, Responses, Anthropic proxy, Gemini proxy, gateway OpenAPI, batches, images, provider routing, retries, MCP, headers.

Kong AI Gateway: license, repository, Helm charts, pricing, AI Proxy, AI Proxy Advanced, open-source AI Proxy formats, AI Rate Limiting Advanced, AI Sanitizer, AI Custom Guardrail, load balancing, AI MCP Proxy, payload logging.

Agent Router: license, repository, installation, CLI, security, quota policy, supported endpoints, API, provider fallback, MCP, tracing. The former address, aigateway.envoyproxy.io, redirects there.

OpenRouter: pricing, FAQ, SSO, SCIM and audit log, workspaces, spend controls, sensitive information, guardrails, compatibility, Anthropic Messages, rerank, your own keys, fallbacks, provider selection, models API, MCP server, data collection.

Cloudflare AI Gateway: pricing, dashboard SSO, SCIM, roles, audit logs, spend limits, guardrails, REST API, Google AI Studio, ElevenLabs, Cohere, your own keys, fallbacks, request handling, MCP server portals, logging.

TensorZero: website, repository, archived on GitHub.

Works with

ToolHow
Claude Code and CodexKaeso is their model provider, so every session is logged and budgeted
The Kaeso plugins for Claude Code and CursorManage Kaeso through its MCP server, from keys to limits; the Claude Code plugin's skill connects the server for you
claude.ai, ChatGPT and any MCP clientCall your MCP servers' tools through a project's MCP portal
The OpenAI, Anthropic, Google Gen AI and ElevenLabs SDKs, in any language, and anything with a base URL settingPoint the base URL at Kaeso; supported providers lists what each provider can do through each API
OpenTelemetry collectors and PrometheusReceive a trace per request and the gateway's metrics (monitoring)

In this repository

PartWhat it is
crates/The gateway, the updater and KPP, the protocol every request is decoded to; kaeso-kpp and kaeso-codecs are on crates.io
apps/consoleThe web console
packages/ui and packages/tokensThe UI library and design tokens behind it, on npm, with a showcase
apps/docsThe documentation at docs.kaeso.app, in English, German and Chinese
deploy/, render.yaml, .railway/ and .do/The Compose package, the Helm chart, the container images and the templates for Render, Railway, DigitalOcean, Fly.io, AWS, Azure, Google Cloud, Coolify, Dokploy and CapRover
plugins/The Claude Code and Cursor plugins for Kaeso's MCP server

Architecture explains how the parts fit, and contributing how to work in them.

Community

License

MIT. The Kaeso name and logo are not covered by the license. All trademarks belong to their owners; Kaeso is not affiliated with any provider, client or product it names, including those in the comparison. Third-party notices, trademarks included, are in NOTICE.md.

No installation packages available.

Remote

streamable-http
https://{host}/mcp