OpenAI-compatible endpoint

The compat endpoint accepts any model name and automatically resolves which provider to route to. It is designed as a drop-in replacement for the OpenAI API — any client built against the OpenAI SDK works without modification by changing only the base_url and api_key.

Use the compat endpoint when you want to:

Switch providers without changing code — point your existing OpenAI SDK client at the gateway and change only the base_url and api_key. Route to Claude, Gemini, or any other provider by changing the model name.
Access multiple providers from one endpoint — your code always calls the same URL; the gateway determines which provider to use.
Use OpenRouter as a fallback — when your gateway has an OpenRouter BYOK key configured, any model identifier that OpenRouter supports will work automatically, even if it is not listed in the built-in model registry of the gateway.

Use the provider-native endpoint instead if you need provider-specific features (such as extended thinking of Anthropic or inference profiles of AWS Bedrock) that are not available through the OpenAI-compatible format.

This page is the single authoritative reference for compat model resolution. The Providers overview page cross-references this page for resolution details.

Endpoint URL

POST /v1/{tenant}/{gateway}/compat/chat/completions

Segment	Description
`{tenant}`	Your tenant identifier
`{gateway}`	Your gateway identifier

The request and response formats are identical to the OpenAI Chat Completions API.

Model resolution

The compat endpoint resolves a provider for each request using a three-tier lookup.

flowchart TD
    A["Incoming request<br/>(model name)"] --> T1{"Tier 1<br/>exact-map match?"}
    T1 -->|"hit (e.g. claude-opus-4-6 → Anthropic)"| R1["Route to mapped provider"]
    T1 -->|miss| T2{"Tier 2<br/>prefix-map match?"}
    T2 -->|"hit (e.g. grok- → xAI)"| R2["Route to prefix provider"]
    T2 -->|miss| T3{"Tier 3<br/>OpenRouter configured?"}
    T3 -->|yes| R3["Route to OpenRouter"]
    T3 -->|no| E400["400 model_not_found"]

Tier 1 — Exact match

The model name is compared against a known model registry. Key examples (not exhaustive):

Model	Provider
`gpt-4o`	`openai`
`gpt-4o-mini`	`openai`
`gpt-4-turbo`	`openai`
`gpt-3.5-turbo`	`openai`
`o1`	`openai`
`o1-mini`	`openai`
`o3-mini`	`openai`
`claude-opus-4-6`	`anthropic`
`claude-sonnet-4-6`	`anthropic`
`claude-haiku-4-5`	`anthropic`
`claude-3-5-sonnet-20241022`	`anthropic`
`claude-3-5-haiku-20241022`	`anthropic`
`claude-3-opus-20240229`	`anthropic`
`gemini-2.0-flash`	`gemini`
`gemini-2.0-flash-lite`	`gemini`
`gemini-1.5-pro`	`gemini`
`gemini-1.5-flash`	`gemini`
`llama-3.3-70b-versatile`	`groq`
`llama-3.1-8b-instant`	`groq`
`mixtral-8x7b-32768`	`groq`
`deepseek-chat`	`deepseek`
`deepseek-reasoner`	`deepseek`
`grok-3`	`xai`
`grok-3-mini`	`xai`
`grok-2-1212`	`xai`
`sonar-pro`	`perplexity`
`sonar`	`perplexity`
`sonar-reasoning-pro`	`perplexity`

Tier 2 — Prefix match

If no exact match is found, the model name prefix is matched:

Prefix	Provider
`gpt`	`openai`
`o1`	`openai`
`o3`	`openai`
`o4`	`openai`
`claude`	`anthropic`
`gemini`	`gemini`
`command`, `embed-`	`cohere`
`anthropic.`, `meta.`, `amazon.`, `mistral.`, `cohere.`, `ai21.`, `stability.`	`bedrock`
`mistral`, `mixtral`, `codestral`	`mistral`
`deepseek`	`deepseek`
`grok`	`xai`
`sonar`	`perplexity`
`meta-llama/`, `deepseek-ai/`, `Qwen/`, `zai-org/`	`together`
`accounts/fireworks/`	`fireworks`
`meta/`, `nvidia/`	`nvidia`
`@cf/`	`cloudflare`
`HuggingFaceH4/`, `tiiuae/`, `bigcode/`, `EleutherAI/`, `microsoft/`, `google/`, `stabilityai/`, `mistralai/`, `sentence-transformers/`	`huggingface`

Tier 3 — OpenRouter fallback

If neither an exact nor a prefix match is found, the request is forwarded to OpenRouter — which supports hundreds of models from many providers under a single API — but only when the gateway has an openrouter provider configured (a stored OpenRouter BYOK key). With OpenRouter configured, any model identifier that OpenRouter accepts will work through the compat endpoint. Without it, an unclassifiable model id is rejected with 400 model_not_found rather than being attributed to a provider the caller never named.

💡 Note: OpenRouter fallback requires a valid OpenRouter API key stored as a BYOK key for the openrouter provider on your gateway.

Routing rules and compat resolution

Routing rules apply on top of compat resolution. If a routing rule matches the incoming request before provider resolution runs, the provider and model configured in the rule are used instead.

Streaming normalisation

All providers use different server-sent event (SSE) formats for streaming responses. The compat endpoint normalises all provider streams into the OpenAI chunk format:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1753718400,"choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1753718400,"choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1753718400,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1753718400,"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":3,"total_tokens":15}}

data: [DONE]

Every data: frame on this wire is an OpenAI-shaped chunk (a :-prefixed SSE comment may appear as a keepalive and is ignored by every conformant reader) — the terminal usage chunk carries an empty choices array, exactly like OpenAI's include_usage chunk, so a client that validates each chunk against the OpenAI schema (@ai-sdk/openai-compatible and everything built on it) accepts the whole stream. The gateway's own aig_* extension frames are sent only to a client that opts in with x-aig-turn-id or x-aig-extensions: 1; see Inference — what a plain client receives.

When token counts are available, a usage chunk is emitted immediately before data: [DONE], even if the upstream provider does not include usage in its stream. This keeps usage-based billing and token tracking consistent across providers. The usage chunk is omitted when both the prompt and completion token counts are zero, or when the client opts out by setting stream_options.include_usage to false.

When a single answer is produced over multiple upstream legs — an automatic length-cap continuation, or a gateway tool loop — the buffered (internally collected) egress reports the sum of every leg's tokens (prompt, completion, and cache-creation/read/deletion) in its single usage chunk, with total_tokens the summed prompt plus completion. A LIVE stream instead emits one usage chunk per leg, so a client that needs the turn total sums them. Either way the totals add up to what the gateway bills and persists, and any model reasoning produced on an earlier leg is carried through in full on the buffered path.

Using the compat endpoint

The following examples show how to call the compat endpoint from common clients.

Python (openai library):

from openai import OpenAI

client = OpenAI(
    base_url="https://your-gateway-host/v1/my-tenant/my-gateway/compat",
    api_key="myra_your_inference_token"
)

response = client.chat.completions.create(
    model="claude-sonnet-4-6",   # any model — provider resolved automatically
    messages=[{"role": "user", "content": "Hello"}]
)

Node.js (openai library):

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://your-gateway-host/v1/my-tenant/my-gateway/compat',
  apiKey: 'myra_your_inference_token',
});

const response = await client.chat.completions.create({
  model: 'gemini-2.0-flash',
  messages: [{ role: 'user', content: 'Hello' }],
});

curl:

curl https://your-gateway-host/v1/my-tenant/my-gateway/compat/chat/completions \
  -H "Authorization: Bearer myra_your_inference_token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'