gpt-realtime

What is gpt-realtime?

OpenAI has introducedgpt-realtime, an advanced speech-to-speech model designed to enhance voice interactions. This model is part of the Realtime API, which is now generally available and optimized for developers and enterprises to build reliable, production-ready voice agents. The API supports remote MCP servers, image inputs, and phone calling through Session Initiation Protocol (SIP), making voice agents more capable and context-aware.

Benefits

gpt-realtimeoffers several key advantages:

• Natural and Expressive Speech: The model produces speech that sounds more natural and expressive, making interactions feel more human-like.
• Improved Instruction Following: It excels at following complex instructions, such as reading disclaimer scripts word-for-word, repeating alphanumerics, and switching languages mid-sentence.
• Enhanced Function Calling: The model can call the right tools at the right time, improving its usefulness in production environments.
• Multi-Language Support: It can switch seamlessly between languages and adapt to different tones and accents.
• Higher Intelligence: The model shows stronger reasoning capabilities and can comprehend native audio with greater accuracy.
• Reduced Latency: Unlike traditional pipelines, the Realtime API processes and generates audio directly through a single model, reducing latency and preserving nuance in speech.

Use Cases

gpt-realtimecan be used in various applications, including:

• Customer Support: Providing natural, empathetic, and accurate responses to customer inquiries.
• Personal Assistance: Assisting with tasks like scheduling, reminders, and information retrieval.
• Education: Offering interactive learning experiences and tutoring.
• Real Estate: Simplifying decisions like buying, selling, and renting a home by making interactions feel as natural as a conversation with a friend.

Pricing

The generally available Realtime API and the newgpt-realtimemodel are available to all developers starting today. Prices forgpt-realtimehave been reduced by 20% compared togpt-4o-realtime-preview, at $32 / 1M audio input tokens ($0.40 for cached input tokens) and $64 / 1M audio output tokens. Fine-grained control for conversation context has been added to let developers set intelligent token limits and truncate multiple turns at a time, significantly reducing cost for long sessions.

Vibes

Since the initial release of the Realtime API in public beta last October, thousands of developers have built with the API and helped shape the improvements being released today. The model has been optimized for reliability, low latency, and high quality to successfully deploy voice agents in production.

Additional Information

gpt-realtimewas trained in close collaboration with customers to excel at real-world tasks like customer support, personal assistance, and education. The model shows improvements across audio quality, intelligence, instruction following, and function calling. Two new voices, Cedar and Marin, are available exclusively in the Realtime API starting today. The existing eight voices are also being updated to benefit from these improvements.

The Realtime API incorporates multiple layers of safeguards and mitigations to help prevent misuse. Developers can easily add their own additional safety guardrails using the documentation. The Realtime API prohibits repurposing or distributing outputs from the services for spam, deception, or other harmful purposes. Developers must also make it clear to end users when they're interacting with AI, unless it's already obvious from the context. The Realtime API uses preset voices to help prevent malicious actors from impersonating others. The Realtime API fully supports General Data Protection Regulation (GDPR) for EU-based applications and is covered by the OpenAI Acceptable Use Policy.