Model failover

How it works

• Avenro first sends your request to the model you name, trying each of its deployments in turn. Most models have one: the provider's API.
• Only if none of them can take it, and every failure was one in which the model could not have run your request, does Avenro try the model's fallback models, in their configured order. A fallback's own fallbacks are never tried.
• A request makes at most 3 attempts in all, fallbacks included, and never tries the same deployment twice. An attempt starts only while at least 5 seconds of the request's time are left, and no further attempt starts once you have disconnected.
• Nothing is retried after a response has started reaching you.

Configured fallbacks

Each fallback is made and served by the same provider as the model it stands in for, so your request is still processed by the company you chose, under its data policy.

Which failures lead to a fallback

Each attempt that fails is put in one category, and only some of them let a fallback model be tried next:

Another deployment of the model you asked for, where it has one, is tried after any failure but a rejection, as before failover existed. Once a failure has ruled fallbacks out, none is tried for that request, whatever fails next.

Which requests can move

A fallback model takes a request only if it can serve it the way you asked:

• It supports everything the request uses: tools (tools, tool_choice, or tool calls in the conversation), JSON output (response_format of type json_object or json_schema), images, streaming and reasoning_effort.
• The request's max_tokens (or max_completion_tokens) is within the fallback's maximum output, and the prompt fits its context window.
• The request uses only these fields: model, messages, stream, stream_options, max_tokens, max_completion_tokens, temperature, top_p, stop, seed, presence_penalty, frequency_penalty, user, n, tools, tool_choice, parallel_tool_calls, response_format and reasoning_effort, with n at 1 and messages made of text and image_url parts (store and metadata, which Avenro never forwards, don't count). Any other field, such as a provider's own extension or logprobs, keeps the request on the model you asked for.
• Its largest possible cost for the request, at its own price, fits within the credits reserved for the model you asked for. A request never costs more because it failed over, and your balance and each API key's spending limit are checked exactly as they would have been.

Billing

• You pay for the model that answered, at its price on the models page, for the tokens it used. That is what x-avenro-cost-usd states and what your usage record shows.
• Failed attempts are never charged.
• Your usage record names the model that answered, so the dashboard's usage and analytics count the request under it; the record also keeps the model you asked for.

Streaming

A stream fails over only before it starts: before any byte of it reaches you. Once it has started, a failure ends it with an error event, as described under streaming; it is never restarted on another model, so no output is repeated.

Response headers

• x-avenro-failover: true when the response came not from the model's first-choice deployment, which failed or was unavailable, but from another deployment of the model or from a fallback model; false otherwise. Sent with x-avenro-execution-path, on every response a deployment produced.
• x-avenro-model: the model that served the request, the one you asked for or its fallback.
• The body's model stays the id you sent, so code that reads it keeps working; compare it with x-avenro-model to see whether a fallback model answered.
• A request keeps one id, x-avenro-request-id, through all its attempts. See execution metadata for every header.

Turning it off

Send the request header x-avenro-failover: off to keep a request on the model you name; another deployment of that same model may still serve it. on is the default, and any other value is refused with a 400.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVENRO_API_KEY"],
    base_url="https://api.avenro.tech/v1",
)

response = client.chat.completions.with_raw_response.create(
    model="claude-fable-5-1",
    messages=[{"role": "user", "content": "Hello"}],
)
completion = response.parse()
print("Requested:", completion.model)
print("Served by:", response.headers.get("x-avenro-model"))
print("Failover:", response.headers.get("x-avenro-failover"))

# Only the model you name, whatever happens:
completion = client.chat.completions.create(
    model="claude-fable-5-1",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={"x-avenro-failover": "off"},
)

When nothing can serve it

If neither the model nor any fallback it could use serves the request, you get 503 upstream_unavailable, as without failover, and are not charged. A fallback model's own error is never passed on to you: the model you asked for never saw the request. See errors.