Adds experimental realtime API support for voice conversations
Vercel AI SDK adds experimental realtime API support for voice conversations across OpenAI, Google, and xAI providers, including a React hook and ephemeral token creation for browser and server use.
Patch Changes ce769dd: feat(provider): add experimental Realtime API support for voice conversations Adds first-class support for realtime (speech-to-speech) APIs: `Experimental_RealtimeModelV4` spec in `@ai-sdk/provider` with normalized event types and factory OpenAI, Google, and xAI realtime provider implementations `openai.experimental_realtime()` / `google.experimental_realtime()` / `xai.experimental_realtime()` work in both server and browser `.getToken()` static method on each provider for server-side ephemeral token creation `experimental_getRealtimeToolDefinitions` helper for provider session tool definitions `experimental_useRealtime` hook in `@ai-sdk/react` returning `UIMessage[]` (aligned with `useChat`), with `onToolCall` and `addToolOutput` for client-driven tool execution `inputAudioTranscription` session config for showing transcribed user audio messages when supported by the provider Updated dependencies [ce769dd] @ai-sdk/provider@4.0.0-canary.18 @ai-sdk/gateway@4.0.0-canary.99 @ai-sdk/provider-utils@5.0.0-canary.46
- github.comVercel AI SDK — ai v7.0.0-canary.165primary
- github.comVercel AI SDK — xai v4.0.0-canary.71
- github.comVercel AI SDK — react v4.0.0-canary.168
- github.comVercel AI SDK — provider v4.0.0-canary.18
§ how this story moved
- primary — Vercel AI SDK — Releases publishes the launch post.
- Vercel AI SDK — Releases picks up coverage.
- Vercel AI SDK — Releases picks up coverage.
- Vercel AI SDK — Releases picks up coverage.