Skip to main content

​
What this guide is

You have an existing OpenAI Realtime API integration and want to point it at Dialog-RSN-1 instead. Since this API is wire-compatible with OpenAI’s Realtime protocol, most of an existing client survives the move untouched. The work is in the differences, not a rewrite.

​
What stays the same

  • The JSON event shape and names: session.update, conversation.item.create, response.create, response.output_text.delta, response.done, and the rest of the vocabulary your client already parses.
  • The add-then-respond turn shape: add an item to the conversation, then ask for a response to it, exactly as before.
  • The function-calling shape: response.function_call_arguments.delta/.done, answered with a function_call_output item and a fresh response.create.

​
What to change

If you use the official openai SDK, the first and third rows are the whole connection change: pass the new base_url and key to the client, and it builds the URL and sends the key as a bearer token, which this API accepts.

Full code on GitHub

A voice agent on the openai SDK, before and after the move to text replies. diff before.py after.py is the whole migration.

​
What to strip out

An OpenAI Realtime client usually carries settings and code this API has no use for. A few leftover settings return errors, and with text replies, leftover audio code waits for events that never arrive. Session settings this API rejects. Each of these returns an unsupported_parameter error naming the field:
  • reasoning and prompt.
  • tool_choice other than "auto", and any tool that isn’t a function, MCP servers included.
  • turn_detection: null, and in turn_detection, idle_timeout_ms and "interrupt_response": false.
  • An audio.output.format type other than audio/pcm.
output_modalities: ["text", "audio"] fails with invalid_request_error. Response and item settings this API rejects. These also return unsupported_parameter or unsupported_item_type, and they’re common in OpenAI clients:
  • In response.create, anything except instructions, max_output_tokens and metadata that differs from the session. That covers out-of-band responses (conversation: "none"), a per-response output_modalities, tools or tool_choice, and input. Change the session with session.update instead.
  • Audio content (input_audio) in conversation.item.create. Stream the caller’s audio with input_audio_buffer.append.
A rejected session.update applies none of its fields, so one leftover setting also throws away the instructions and tools sent alongside it. If your client logs errors and carries on, it ends up running with no prompt and no tools, so remove the fields rather than silencing the error. Session settings this API ignores. These are accepted and dropped, so session.updated shows only what’s in effect. Remove them anyway, since they do nothing here:
  • model, tracing, truncation and include.
  • The turn_detection timing fields threshold, prefix_padding_ms and silence_duration_ms, and "interrupt_response": true. "type": "semantic_vad" leaves the session on server_vad.
  • audio.input.noise_reduction.
  • An OpenAI voice name such as marin in audio.output.voice, and audio.output.speed.
This is what this API acts on, for text replies:
what this service acts on
audio.input.transcription is accepted, but there’s no transcription model to pick: Dialog-RSN-1 transcribes the caller as part of its own turn, and the transcript arrives as the usual conversation.item.input_audio_transcription.* events. Because it comes from the model’s turn, .completed arrives only when that response completes. A reply cancelled by a barge-in or response.cancel, or one that fails, gets at most some .delta events and no .completed. With the managed voice, .completed lands once the spoken reply has finished playing. Turn timing isn’t tunable because the turn boundary here is a model judgment, not a silence timer. Dialog-RSN-1 confirms or overrules what the detector proposes, based on the conversation. What you’d have expressed as silence_duration_ms you express in instructions, in plain language, as the prompt guide explains. The fixed thresholds behind the detector are in the reference. The audio playback path. A real OpenAI Realtime client typically listens for response.output_audio.delta and feeds it to a jitter-buffered player. Dialog-RSN-1 writes its reply as text, and you choose who speaks it:
  • Your own text-to-speech. Set output_modalities: ["text"], delete the playback code, and read the reply from response.output_text.delta instead. No response.output_audio.* events arrive. This is what the companion repo’s after.py does.
  • The managed voice. Sessions start with output_modalities: ["audio"], and the service speaks each reply itself. The audio arrives as response.output_audio.delta, PCM16 at 24 kHz, with the transcript in response.output_audio_transcript.*, so your playback code keeps working. To choose a voice, pass a Cartesia voice ID in audio.output.voice.
Echo avoidance. With text replies, if your client mutes the microphone while the assistant’s audio plays, to stop the assistant hearing itself through the speakers, remove that too. This API produces no audio to leak. When you add your own text-to-speech, report playback instead, as the streaming to TTS guide covers.

​
Known limitations to account for

If your client was built against OpenAI’s fuller feature set, you’re more likely to notice the gaps than if you’d built fresh against this API. Two worth knowing before you migrate: with text replies, conversation.item.truncate is acknowledged but doesn’t actually shorten history, and MCP server tools are rejected rather than half-supported. The full list is in Known limitations.

Overview

What realtime streaming is, and how turn-taking, generation, and prompting fit together.

Reference

Every event, the session object, function calling, and error codes.

Python from scratch

Building a client against this API directly, without carrying over an existing OpenAI integration.
Last modified on October 14, 2026