Upstream status
Not published to Relays
vLLM does not publish a status feed Relays can read, so no availability is reported here.
One alias in front of vLLM.
Your application calls a Relays alias. Relays owns the connection to your vLLM endpoint, applies the route policy, and hands back the provider response in the shape your SDK expects.
- Your application
Keeps the OpenAI-compatible request it already sends.
- Relays
Holds the key, picks the candidate, writes the receipt.
- vLLM
Receives the call on its own base URL.
- OpenAI-compatible
- Chat Completions
- No upstream key by default
http://localhost:8000/v1- Maintained Relays preset
Relays’ vLLM preset targets a local OpenAI-compatible server with no upstream authentication by default. Use an explicit registry entry when the endpoint or authentication differs.
What is measured, and what is not.
Availability comes from vLLM. Latency, error rate, and cost per successful request come from your own traffic, so Relays leaves them empty until requests run.
Relays measurements
No sample yet
Time to first token, error rate, and cost per successful request are read from requests that ran through Relays. This route has none, so there is nothing to plot.
Models at the edge.
Model IDs sit behind a Relays alias, so a catalogue change at vLLM never reaches your code.
Catalogue not connected
Relays has not published a vLLM model list on this page. Route candidates are still set by model ID in your route configuration.
Every route carries a policy.
A route is a list of candidates, a budget, and a record of what happened. You set all three before any traffic runs.
Candidates stay inside the contract.
vLLM can be part of an explicit Chat Completions route alongside hosted candidates. Route priority makes a local-first policy visible and reviewable.
See how a route is chosenA ceiling on this route.
Set an alert threshold and a stop threshold for the route rather than for the whole application. Relays checks both before the call goes upstream.
Each attempt is written down.
Every call records the alias, the candidates tried, the status each returned, tokens in and out, and the estimated cost of the attempt that served it.
Read a route receipt
One endpoint is enough to start.
Keep the client you already wrote. Bring us the workload and we will map the first vLLM route with you.