AI Gateway now supports per-inference timeouts for faster failover than the provider default. If a provider doesn't start responding within your configured timeout, AI Gateway aborts the request and falls back to the next available provider.provider Provider timeouts are available in beta for BYOK (Bring Your Own Key) credentials only, with support for system provider timeouts coming soon. Note that some providers don't support stream cancellation, so you may still be charged for timed-out requests depending on the provider.

Basic usage Set timeouts per provider in milliseconds using in .providerTimeoutsproviderOptions.gateway Advanced usage with multiple providers and failover Use with to control both the provider sequence and failover speed.order For more information, read the custom provider timeouts .documentation Read more