Qwen-Audio-3.0-Realtime real-time voice conversation iOS SDK

User guide: For model introductions and selection advice, see Real-time voice conversation.

Quick start

Obtain an API key: Obtain and configure an API key.
Download the SDK and run the sample code:
- Download the latest SDK package.
- Extract the ZIP package and add nuisdk.xcframework to your project.
- In Build Phases > Link Binary With Libraries, add nuisdk.xcframework.
- In General > Frameworks, Libraries, and Embedded Content, set nuisdk.xcframework to Embed & Sign.
- Open the project in Xcode. The sample code is in DashQwenAudioChatViewController.m. Replace the API key to try the feature.

Call procedure

Initialize the SDK.
Set parameters for your use case. Use the parameters argument of nui_initialize to set the connection and control parameters, and use nui_set_params to set the voice conversation parameters.
Call nui_dialog_start to start the conversation.
In onNuiAudioStateChanged, start the recording device based on the audio state.
Continuously supply recording data in onNuiNeedAudioData, or call nui_update_audio_data to actively push recording data.
Continuously receive the audio returned by the model in onNuiAssistEventCallback.
Listen for events and obtain event information in onNuiEventCallback.
Call nui_dialog_cancel to stop the conversation, and listen for EVENT_TRANSCRIBER_COMPLETE to confirm that it has ended.
When the conversation feature is no longer needed, call nui_release to release the SDK resources.

Request parameters

Connection and control parameters

Pass a JSON string in the parameters argument of nui_initialize. Example: The following JSON string does not list every parameter. Add parameters as needed for your use case.

{
  "url": "wss://dashscope.aliyuncs.com/api-ws/v1/inference",
  "apikey": "st-****",
  "device_id": "my_device_id",
  "service_mode": "1"
}

Parameters

Parameter	Type	Required	Description
`url`	`String`	Yes	Service endpoint: `wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` China (Beijing): `wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` Singapore: `wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` Replace `{WorkspaceId}` with your actual Workspace ID.
`apikey`	`String`	Yes	API key.
`service_mode`	`String`	Yes	Runtime mode. Set this parameter to `"1"` for real-time voice conversation.
`device_id`	`String`	Yes	A unique string that identifies the end user. You can use an in-app user ID or a client-generated device identifier. This ID is mainly used for log tracing and troubleshooting.
`audio_update_manually`	`String`	No	Whether to actively push audio data. Default: `"false"`. If set to `"true"` and the SDK supports on-device audio capabilities such as AEC and VAD, those capabilities are enabled by default.
`workspace`	`String`	No	The storage path for on-device resource files. This parameter is required when `audio_update_manually` is `"true"` and an on-device audio capability such as AEC or VAD is enabled.
`debug_path`	`String`	No	The log file storage path. This parameter takes effect only when `save_log` is `true` in `nui_initialize`. In this case, the path is required. The SDK retains at most two log files locally.
`save_wav`	`String`	No	Whether to save debug audio under `debug_path`. Default: `"false"`. Valid values are `"true"` and `"false"`. This parameter takes effect only when `save_log` is `true`, and `debug_path` must also be set.
`max_log_file_size`	`int`	No	The maximum log file size in bytes. This parameter takes effect only when `save_log` is `true`. Default: `104857600` (100 × 1024 × 1024 bytes, or 100 MiB).
`log_track_level`	`int`	No	The filter level for logs sent through `onNuiLogTrackCallback`. Default: `2`. Valid values: `0` (VERBOSE), `1` (DEBUG), `2` (INFO), `3` (WARNING), `4` (ERROR), and `5` (NONE). A log is returned only when its level is greater than or equal to both `log_track_level` and the `level` passed to `nui_initialize`. For example, if `log_track_level` is `2` (INFO) and `level` is `3` (WARNING), only WARNING and higher-level logs (values greater than or equal to `3`) are returned.

Voice conversation parameters

Pass a JSON string in the params argument of nui_set_params. Example: The following JSON string does not list every parameter. Add parameters as needed for your use case.

{
  "service_type": 4,
  "nls_config": {
    "model": "qwen-audio-3.1-realtime-plus",
    "sr_format": "pcm"
  }
}

Parameters

Top-level parameter	Type	Required	Description
`service_type`	`int`	Yes	Voice service type. Set this parameter to `4` for real-time voice conversation.
`nls_config`	`object`	Yes	Core voice conversation configuration, including model selection and conversation behavior.
`nls_config.model`	`string`	Yes	Model name. Supports the `qwen-audio-3.1-realtime-plus`, `qwen-audio-3.0-realtime-plus`, and `qwen-audio-3.0-realtime-flash` model series.
`nls_config.sr_format`	`string`	Yes	Input audio format. Only `pcm` is supported. The default format is 16 kHz, 16-bit, mono PCM.
`nls_config.modalities`	`string`	No	A string containing an array of output modalities. Valid values: `["text"]`: returns text only. `["audio", "text"]` (default): returns both audio and text.
`nls_config.voice`	`string`	No	TTS voice. The default is `longanqian_v3.1` for 3.1 Plus and `longanqian` for 3.0 Plus/Flash. This parameter can be set only in the first `session.update`; later values are ignored. System voices: `longanqian`, `longanlingxin`, `longanlingxi`, `longanxiaoxin`, and `longanlufeng`. You can also specify a cloned `voice_id` created through the voice cloning API. See Voice configuration.3.1 Plus also supports `longanqian_v3.1`, `longanhuan_v3.1`, `longanlingxin_v3.1`, `longanfengyue_v3.1`, `xunanchuan_v3.1`, `beth_v3.1`, `betty_v3.1`, `cally_v3.1`.
`nls_config.enable_speech_emotion`	`BOOL`	No	Whether to enable enhanced emotional expression. When enabled, the response voice has more noticeable emotional variation. Default: `YES`. Valid values: `YES` and `NO`.
`nls_config.instructions`	`string`	No	System instructions that define the model's role, response style, and behavioral preferences for the entire session.
`nls_config.max_history_turns`	`int`	No	Maximum number of historical question-answer turns allowed in a request. Valid values: 1 to 50. Default: `20`.
`nls_config.tools`	`string`	No	A string containing an array of Function Calling tool definitions. After you configure this parameter, the model decides whether to call a tool based on the user input. Each definition uses `type` (required and fixed to `function`) and a `function` object containing `name` (required), `description` (optional), and `parameters` (optional). If a function has no arguments, omit `parameters`. Example: `[ { "type": "function", "function": { "name": "get_weather", "description": "Queries weather information for a specified city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City" } }, "required": [ "city" ] } } } ]`
`nls_config.turn_detection`	`string`	No	A string containing a JSON object for turn detection. If omitted, the session uses push-to-talk mode, in which audio is committed and inference is triggered manually. If set, duplex conversation mode is enabled.
`nls_config.turn_detection.type`	`string`	No	VAD type. `server_vad` (default) detects speech boundaries from acoustic features and automatically triggers inference. `smart_turn` combines acoustic and semantic signals; sounds without semantic content, such as filler sounds, do not start a turn or interrupt model playback.
`nls_config.turn_detection.threshold`	`float`	No	VAD sensitivity. This parameter applies only to `server_vad` and has no effect on `smart_turn`. Lower values make VAD more sensitive to quiet sounds and background noise; higher values require clearer and louder speech. Valid range: `[-1.0, 1.0]`. Default: `0.5`.
`nls_config.turn_detection.silence_duration_ms`	`int`	No	Minimum silence duration after speech, in milliseconds, before the model response is triggered. This parameter applies only to `server_vad` and has no effect on `smart_turn`. Lower values reduce latency but can trigger on brief pauses. Valid range: `[200, 6000]`. Default: `800`. Recommended for conversation: `400` to `800`.
`nls_config.turn_detection.voiceprint_audio_urls`	`string`	No	A string containing an array of publicly accessible prerecorded audio URLs for the target speaker. This parameter applies only to `smart_turn`. In duplex conversations, it helps the model focus on the target speaker and ignore other speakers and background noise. Up to five URLs are supported. Audio must be 16 kHz PCM or WAV.

Key APIs

NeoNui

nui_initialize

Initializes the voice conversation SDK instance. The SDK is a singleton. Do not initialize it again before you call nui_release. Method signature

-(NuiResultCode) nui_initialize:(const char *)parameters
                       logLevel:(NuiSdkLogLevel)level
                        saveLog:(BOOL)save_log;

Parameter descriptions

Parameter	Type	Description
`parameters`	`char*`	A JSON string that contains authentication, connection, and debugging parameters. See Connection and control parameters.
`level`	`NuiSdkLogLevel`	Print level for the SDK's own logs.
`save_log`	BOOL	Whether to save logs locally. If set to `YES`, specify a path with `debug_path` in Connection and control parameters, and optionally set the file size with `max_log_file_size`.

Return value Returns an error code. See Error code reference.

nui_set_params

Sets the Voice conversation parameters in JSON format. Call this method before nui_dialog_start. Method signature

-(NuiResultCode) nui_set_params:(const char *)params;

Parameter descriptions

Parameter	Type	Description
`params`	`char*`	Voice conversation parameters.

Return value Returns an error code. See Error code reference.

nui_dialog_start

Starts the conversation. Method signature

-(NuiResultCode) nui_dialog_start:(NuiVadMode)vad_mode
                      dialogParam:(const char *)dialog_params;

Parameters

Parameter	Type	Description
`vad_mode`	`NuiVadMode`	VAD mode. Fixed to `MODE_P2T`.
`dialog_params`	`char*`	If `apikey` in the connection and control parameters is a temporary API key, update it here after it expires. JSON format: `{ "apikey": "st-****" }`

Return value An error code. See Error code reference.

nui_dialog_cancel

Ends the conversation or immediately cancels the current interaction. Method signature

-(NuiResultCode) nui_dialog_cancel:(BOOL)force;

Parameter descriptions

Parameter	Type	Description
`force`	`BOOL`	Whether to end forcibly and discard the final result. `YES`: End the task immediately without waiting for the server to return the final conversation result. `NO`: End the task, but wait for the complete result to return.

Parameter

Type

Description

force

BOOL

Whether to end forcibly and discard the final result.

YES: End the task immediately without waiting for the server to return the final conversation result.
NO: End the task, but wait for the complete result to return.

Return value Returns an error code. See Error code reference.

nui_dialog_action

Sends a conversation action command during an interaction to update runtime behavior such as the conversation context. Method signature

- (NuiResultCode) nui_dialog_action:(const char *)params;

Parameters

Parameter	Type	Description
`params`	`char*`	A JSON string used to update runtime behavior such as the conversation context.
`params.type`	`String`	Set to `"action"`.
`params.command`	`String`	Runtime command. Valid values: `function_call`: updates a function call request. `play_start`: when on-device AEC is used, notifies the SDK that audio playback has started. `play_over`: when on-device AEC is used, notifies the SDK that audio playback has ended.
`params.context`	`String`	Function call request update. Used when `command` is `"function_call"`.
`params.context.type`	`String`	Event type. This parameter is required when `command` is `"function_call"`. `conversation.item.create` inserts a conversation item for historical context, supplemental text, or a tool result. After sending `conversation.item.create`, use `response.create` to trigger another inference.
`params.context.item`	`object`	Required when `params.context.type` is `conversation.item.create`. The conversation item to create.
`params.context.response`	`object`	Optional when `params.context.type` is `response.create`. Overrides the session defaults for this inference. If omitted, the current session configuration is used.

context.item parameters:

Parameter	Type	Description
`id`	`String`	Optional unique conversation item ID. If omitted, the server generates one. An error is returned if the specified ID already exists.
`type`	`String`	Required item type. Valid values: `message`: a regular message. `function_call`: a function call request. This type is typically generated by the server. The client can also use it to add historical context. `function_call_output`: a tool execution result. After receiving a `function_call`, the client runs the tool and uses this type to write back the result.
`role`	`String`	Required for `message`. Valid values: `system`, `user`, and `assistant`.
`content`	`array`	Required for `message`. Each item contains a `type` and the associated data field. `system` supports `input_text` with `text`; `user` supports `input_text` with `text` and `input_audio` with Base64-encoded `audio`; `assistant` supports `output_text` with `text`.
`call_id`	`String`	Required for `function_call` and `function_call_output`. The unique ID that associates a function call request with its result.
`name`	`String`	Required for `function_call`. The name of the function to call.
`arguments`	`String`	Required for `function_call`. Function arguments as a JSON string.
`output`	`String`	Required for `function_call_output`. Tool execution result as a JSON string.

context.response parameters:

Parameter	Type	Description
`modalities`	`array`	Output modalities. `["text"]` returns text only. `["audio", "text"]` (default) returns both audio and text.
`voice`	`string`	Overrides the TTS voice for this inference.

Example:

{
  "type": "action",
  "command": "function_call",
  "context": {
    "item": {
      "call_id": "call_xxxx",
      "output": "{\"city\":\"Hangzhou\",\"condition\":\"sunny\",\"temperature\":18}",
      "type": "function_call_output"
    },
    "type": "conversation.item.create"
  }
}

{
  "type": "action",
  "command": "function_call",
  "context": {
    "response": {
      "modalities": ["text", "audio"]
    },
    "type": "response.create"
  }
}

Return value An error code. See Error code reference.

nui_update_audio_data

When audio_update_manually is set to "true", recording data is no longer supplied through onNuiNeedAudioData. Use this method to actively push it instead. Method signature

-(NuiResultCode) nui_update_audio_data:(const char *)data
                                    Len:(int)length
                              FirstPack:(BOOL)first_pack;

Parameter descriptions

Parameter	Type	Description
`data`	`const char *`	The audio data to push.
`length`	`int`	The length of the audio data, in bytes.
`first_pack`	`BOOL`	You do not need to use this parameter.

Return value Returns an error code. See Error code reference.

nui_push_reference_data

When audio_update_manually is set to "true" and on-device acoustic echo cancellation (AEC) is enabled, use this method to push the audio played by the player as the reference signal. Method signature

-(NuiResultCode) nui_push_reference_data:(const char *)data
                                     Len:(int)length
                               FirstPack:(BOOL)first_pack;

Parameter descriptions

Parameter	Type	Description
`data`	`const char *`	The audio data to push.
`length`	`int`	The length of the audio data, in bytes.
`first_pack`	`BOOL`	You do not need to use this parameter.

Return value Returns an error code. See Error code reference.

nui_release

Releases all internal SDK resources and forcibly terminates all running tasks. After you call this method, the SDK instance becomes unavailable. To use it again, you must call nui_initialize to initialize it again. Method signature

-(NuiResultCode) nui_release;

Return value Returns an error code. See Error code reference.

nui_get_version

Gets the current SDK version. This method returns a value only after nui_initialize is called. Method signature

-(const char*) nui_get_version;

Return value The current SDK version.

nui_get_all_response

Gets the complete information for the current event callback. Method signature

-(const char*) nui_get_all_response;

Return value The complete event information as a JSON string.

NeoNuiSdkDelegate: callback listeners

onNuiEventCallback: listen for event information

Method signature

-(void) onNuiEventCallback:(NuiCallbackEvent)nuiEvent
                    dialog:(long)dialog
                 kwsResult:(const char *)wuw
                 asrResult:(const char *)asr_result
                  ifFinish:(BOOL)finish
                   retCode:(int)code;

Parameter descriptions

Parameter	Type	Description
`nuiEvent`	NuiCallbackEvent	Callback event.
`dialog`	`long`	Session ID. You do not need to use this parameter.
`wuw`	`char*`	Voice wake-up. You do not need to use this parameter.
`asr_result`	`char*`	Speech recognition result.
`finish`	`BOOL`	Whether the current recognition round has finished.
`code`	`int`	Error code. Valid when the EVENT_ASR_ERROR event occurs. See Error code reference.

onNuiAudioStateChanged: listen for the audio state

The SDK uses this callback to notify you when to start or stop recording. Method signature

-(void) onNuiAudioStateChanged:(NuiAudioState)state;

NuiAudioState states

Parameter	Description
`STATE_OPEN`	The interaction has started. You can open the recording device and start recording.
`STATE_PAUSE`	The interaction has stopped. You can stop recording.
`STATE_CLOSE`	The SDK instance has been released. You can close the recording device completely.

onNuiNeedAudioData: supply audio data

After the conversation starts, this callback is triggered continuously. Supply the audio data in this callback. You do not need to use this callback when audio_update_manually is set to "true". Method signature

-(int) onNuiNeedAudioData:(char *)audioData length:(int)len;

Parameter descriptions

Parameter	Type	Description
`audioData`	`char *`	The audio data to supply.
`len`	`int`	The size of the supplied audio data, in bytes.

onNuiAssistEventCallback: receive auxiliary data and information

Receives auxiliary events and related data from the SDK. Method signature

-(void) onNuiAssistEventCallback:(NuiCallbackEvent)nuiEvent
                            info:(char*)info
                         infoLen:(int)info_len
                          buffer:(char*)buffer
                             len:(int)len;

Parameter descriptions

Parameter	Type	Description
`nuiEvent`	`NuiCallbackEvent`	The callback event.
`info`	`char *`	You do not need to use this parameter.
`info_len`	`int`	You do not need to use this parameter.
`buffer`	`char *`	Auxiliary data, such as TTS audio returned by the model.
`len`	`int`	The length of the auxiliary data, in bytes.

onNuiLogTrackCallback: listen for tracking logs

This callback receives the SDK's detailed internal logs to help you locate and debug issues.

-(void) onNuiLogTrackCallback:(NuiSdkLogLevel)level
                   logMessage:(const char *)log;

`NuiCallbackEvent`: event types

Event	Description
`EVENT_TRANSCRIBER_STARTED`	The task started successfully.
`EVENT_VAD_START`	Triggered immediately after the task starts. This does not mean that the start of speech has been detected.
`EVENT_VAD_END`	The end of speech was detected.
`EVENT_ASR_PARTIAL_RESULT`	An intermediate speech recognition result.
`EVENT_ASR_ERROR`	An error occurred during the voice conversation.
`EVENT_MIC_ERROR`	Triggered when no audio data is received for two consecutive seconds.
`EVENT_SENTENCE_END`	The end of a sentence was detected and a complete recognition result was returned.
`EVENT_TRANSCRIBER_COMPLETE`	The voice conversation ended.
`EVENT_AUDIO_TRANSCRIPTION`	An incremental text transcript event for audio output. Transcript segments are returned in streaming mode.
`EVENT_AUDIO_TRANSCRIPTION_COMPLETED`	The transcript for audio output is complete.
`EVENT_OTHER_RESULT`	Other event information, such as a Function Calling result.
`EVENT_ASR_TTS_START`	The model started returning TTS audio.
`EVENT_ASR_TTS_DATA`	TTS audio returned by the model.
`EVENT_ASR_TTS_COMPLETE`	The model finished returning TTS audio.
`EVENT_AEC_DATA`	Audio data processed by AEC.

​Quick start

​Call procedure

​Request parameters

​Connection and control parameters

​Voice conversation parameters

​Key APIs

​NeoNui

​nui_initialize

​nui_set_params

​nui_dialog_start

​nui_dialog_cancel

​nui_dialog_action

​nui_update_audio_data

​nui_push_reference_data

​nui_release

​nui_get_version

​nui_get_all_response

​NeoNuiSdkDelegate: callback listeners

​onNuiEventCallback: listen for event information

​onNuiAudioStateChanged: listen for the audio state

​onNuiNeedAudioData: supply audio data

​onNuiAssistEventCallback: receive auxiliary data and information

​onNuiLogTrackCallback: listen for tracking logs

​NuiCallbackEvent: event types

Quick start

Call procedure

Request parameters

Connection and control parameters

Voice conversation parameters

Key APIs

NeoNui

nui_initialize

nui_set_params

nui_dialog_start

nui_dialog_cancel

nui_dialog_action

nui_update_audio_data

nui_push_reference_data

nui_release

nui_get_version

nui_get_all_response

NeoNuiSdkDelegate: callback listeners

onNuiEventCallback: listen for event information

onNuiAudioStateChanged: listen for the audio state

onNuiNeedAudioData: supply audio data

onNuiAssistEventCallback: receive auxiliary data and information

onNuiLogTrackCallback: listen for tracking logs

`NuiCallbackEvent`: event types