Automatic Retries

Automatically retry failed LLM requests with exponential backoff.

Available on all Portkey plans.

• Up to 5 retry attempts
• Trigger on specific error codes
• Exponential backoff to prevent overload
• Optionally respect provider’s Retry-After headers

Examples

Basic (5 attempts)

{
  "retry": { "attempts": 5 },
  "override_params": { "model": "@openai-prod/gpt-4o" }
}

The @provider-slug/model-name format automatically routes to the correct provider. Set up providers in Model Catalog.

Retry on Specific Error Codes

Default retry codes: [429, 500, 502, 503, 504]

Override with on_status_codes:

{
  "retry": { "attempts": 3, "on_status_codes": [408, 429, 500] },
  "override_params": { "model": "@openai-prod/gpt-4o" }
}

When on_status_codes is set, retries trigger only on those codes—not the defaults.

Respect Provider Retry Headers

Enable use_retry_after_headers to use the provider’s retry-after-ms, x-ms-retry-after-ms, or retry-after headers instead of exponential backoff.

{
  "retry": { "attempts": 3, "on_status_codes": [429], "use_retry_after_headers": true },
  "override_params": { "model": "@openai-prod/gpt-4o" }
}

• Falls back to exponential backoff if headers aren’t present
• Cumulative retry wait capped at 60 seconds
• Single Retry-After value > 60s fails immediately

Exponential Backoff

Attempt	Wait Time
Initial	Immediate
1st retry	1 second
2nd retry	2 seconds
3rd retry	4 seconds
4th retry	8 seconds
5th retry	16 seconds

Retry Attempt Header

Check x-portkey-retry-attempt-count in responses:

Value	Meaning
-1	All retries exhausted, request failed
0	No retries configured
>0	Successful on this retry attempt

Retry attempts aren’t logged individually. Response times are summed in a single log entry.