LLM router

Route by the order you approved.

Define capability-aware routes and an ordered provider fallback outside your application. Relays walks that order, takes the first route that clears your rules, and leaves a receipt for the decision.

Open the workbench Browse providers

The workbench

Cheaper routes first. Only if they qualify.

You order the routes. Relays takes the first one that clears your capability and policy rules, and says so plainly when nothing does. Move a route or change a rule and the decision moves with it.

Route policyIllustrative
Workload

Summarise a support thread. Needs tool calling.

Routes are pinned to a region when you define them. A route outside the rule is never attempted.

Simulate a 429 from

Rate-limited routes are attempted, fail, and hand off to the next eligible route.

Your approved order

Reorder to change the decision
  1. zhipu/glm-5.3tool calling · structured output · EU pin

    SelectedFirst eligible route in your order

  2. moonshot/kimi-k3tool calling · structured output · long context · EU pin

    Not reachedThe order settled before this route

  3. openai/gpt-5.6-lunatool calling · vision · structured output · US pin

    Not reachedThe order settled before this route

  4. anthropic/sonnet-5tool calling · vision · structured output · long context · reasoning · US pin

    Not reachedThe order settled before this route

  5. openai/gpt-5.6-soltool calling · vision · structured output · long context · reasoning · US pin

    Not reachedThe order settled before this route

  6. anthropic/fable-5tool calling · vision · structured output · long context · reasoning · US pin

    Not reachedThe order settled before this route

Route receipt

Synthetic

Selected

zhipu/glm-5.3

First route in your order that cleared every capability and policy rule.

Request
Summarise a support thread
Attempts
1
Skipped by rule
0
Tokens
1,840 in · 512 out
Region pin
EU

Selected zhipu/glm-5.3 after 1 attempt and 0 skipped routes.

Synthetic policy, synthetic figures. Nothing on this page reaches a provider.

Fallback

Fallback without the rewrite.

When a provider fails or rate-limits, the next eligible route runs. You approved the order before the incident, so nothing in your application changes during one.

  1. RequestOne client call

    Your SDK, your model alias, unchanged.

  2. Attempt 1zhipu/glm-5.3

    Failed. Rate limited, 429.

  3. Attempt 2moonshot/kimi-k3

    Accepted. Next eligible route in the order.

  4. ResponseSame shape, same stream

    No client rewrite, no redeploy.

Illustrative walk · synthetic providers and statuses

The handoff

The second attempt parses like the first.

A fallback changes the provider, not the response contract. The code that parsed attempt one parses attempt two, with no branch for which route answered. Streaming, tool calls, reasoning and prompt caching all survive the handoff.

First attempt returns

  • Response shape
  • Streaming
  • Tool calls
  • Reasoning
  • Prompt caching
Same contract

Second attempt returns

  • Response shape
  • Streaming
  • Tool calls
  • Reasoning
  • Prompt caching

Evidence

Every route leaves evidence.

A route is only defensible if you can read it back. Every decision, attempt, token count and estimated cost stays attached to the request that caused it.

Decision
Which route ran and the rule that put it first.
Attempts
Every route that was called, in order, with the status it returned.
Cost
Tokens and estimated cost, tied to the request that caused them.
Route receiptSynthetic
request
req_8f2c41d0
workload
Refactor an auth middleware
policy
long context · tool calling
  1. zhipu/glm-5.3Skipped, no long context
  2. moonshot/kimi-k3Failed, rate limited 429
  3. anthropic/sonnet-5Selected, 200 accepted
tokens
31,204 in · 2,916 out
est. cost
$0.012
How the gateway works See pricing

Move the route out of your application.

Bring us the workload and the constraints. We will define the first route with you.

Start a conversation