Qwen-Omni-Realtime AOQ access
Scope of Application
This document is aimed at Qwen-Omni-Realtime. Existing integration documentation covers: qwen3.8-omni-flash-realtime, qwen3.5-omni-plus-realtime, qwen3.5-omni-flash-realtime. The example model is qwen3.8-omni-flash-realtime.
- Use the API Key and Workspace ID corresponding to your selected region and workspace.
-
Beijing domain:
{WorkspaceId}.cn-beijing.maas.aliyuncs.com; Singapore domain:{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com. - Store the API Key only on the server. Do not embed it in a browser or mobile application.
- When capturing audio with a microphone or video with a camera, request the necessary permissions according to platform requirements.
- Download the SDK for your platform from SDK downloads.
Authentication and connection credentials
Send the following request from your AppServer. First configure the API Key as an environment variable. Replace {WorkspaceId} in the domain with your workspace ID. For another region, use the corresponding domain listed above. clientIp is optional. If omitted, the gateway uses the IP address from which it receives the request.
-d '{}' in the request body with -d '{"clientIp":"<CLIENT_PUBLIC_IP>"}'.
Request fields
Configuration | Value | Description |
|---|---|---|
endpoint | Select the endpoint domain for your use case | Specify the endpoint domain. For details, see Select a region, deployment scope, and endpoint. If you use Token Plan, set |
Content-Type |
| Message media type |
Authorization |
| Your API Key |
x-dashscope-rtc-transport |
| Select the AOQ protocol |
clientIp | The client public IP address | Optional. If omitted, the service uses the IP address from which the gateway receives the request. If specified, it uses clientIp. The service selects the best Relay endpoint based on the client IP address. |
Response example
Response fields
Field | Description |
|---|---|
sid | Unique session identifier. |
aoqTokenForClient | Client connection token. Pass this value to the SDK token field. |
clientRelayEndpoints | Array of Relay endpoints. Each element in the response example contains the following fields:
Properties endpointRelay endpoint address.portRelay endpoint port.route_indexThe iOS example below uses the array index when this field is absent. |
clientRelayCertFingerprint | Relay TLS certificate fingerprint. |
sidExpiresInSecs | Session expiration time in seconds. |
extraInfo | Additional information.
Properties workspaceIdHashWorkspace ID hash. |
AOQ Client SDK connection example
clientIp is an optional field in the request body. If not specified, the IP address that requests the Model Studio gateway is used as the client IP. If specified, the clientIp value takes precedence. Have your business AppServer obtain the client's real IP address and pass it in to get the best Relay access point.Workflow

- Obtain connection credentials through AppServer.
- Create the SDK engine and register callbacks.
- Configure media and Data tracks, and disable upstream media transmission.
- Establish the AOQ connection and wait for it to connect.
- Configure the model according to its protocol. Enable the required upstream media after confirmation.
- Disconnect and release resources when interaction is complete.
- AOQ SDK handles media capture, processing, transmission, reception, and playback. Your application must send and receive Data messages according to the target model's event definitions.
-
A successful connection indicates that the transmission channel to the model has been established; the client should enable the required upstream media only after the model initialization is complete. For the example model
qwen3.8-omni-flash-realtimeon this page, wait forsession.updatedafter sendingsession.update, then begin media transmission.
1. Obtain connection credentials
The client requests credentials from your AppServer. The AppServer calls this model's Realtime connection endpoint and includes the API Key and x-dashscope-rtc-transport: moq in the gateway request. For the full endpoint, request, and field descriptions, see Token authentication.
Map the response to AoqConnectConfig:
| Gateway response | SDK configuration |
|---|---|
aoqTokenForClient | token |
sid | sid |
clientRelayCertFingerprint | certFingerprint |
clientRelayEndpoints | relayEndpoints, convert each access point according to the structure |
extraInfo contains workspaceIdHash | workspaceIdHash |
2. Initialize the SDK and register callbacks
Create the engine and register callbacks for connection status, Data messages, and errors. The following example uses Swift on iOS:
AoqEngineDelegate; maintain connection status in onConnectionStatusChange; parse target model events in onDataMsg; handle SDK errors in onError. Error events returned by the model also need to be handled separately within the Data message processing logic. For details on the SDK interface and error descriptions, see Platforms SDK and Feature Reference.
3. Configure the SDK
Select media directions
Directions are relative to the client. Configure only the input and output modalities that your application needs. Do not add tracks for unused directions.
| Direction | Configuration | Typical use |
|---|---|---|
| pushes audio | publishes the Audio track, sets the audio encoder, configures built-in or external capture | voice conversation |
| pulls audio | subscribes to the Audio track, sets the audio decoder, configures the player or external playback | voice response |
| pushes video | publishes the Video track, configures the camera or external video input and encoding parameters | sends video to models that support visual input |
| pulls video | this model does not support video output | do not configure video subscription |
Media directions for this model
The table below lists the media directions for this model. See Applicable Scope for the model’s range; for other integration methods, refer to Qwen-Omni-Realtime.
| Model or application | Send audio | Receive audio | Send video | Receive video |
|---|---|---|---|---|
| Qwen-Omni-Realtime | enable when voice input is required | enable when voice response is needed | enable when visual input is required | cannot enable |
Configure media parameters and tracks
Configure upstream and downstream audio separately: use setAudioEncoderConfig for upstream audio and setAudioDecoderConfig for downstream audio. These parameters control network transmission; the gateway converts audio to the format required by the model. Configure transport encoding and the audio format in model events according to the SDK and target model requirements, respectively.
Start startAudioCapture, startAudioPlayer, or startVideoCapture as needed. For custom input and output, see Advanced features below.
publishTracks are sent by the client, and subscribeTracks are received by the client:
| Scenario | publishTracks | subscribeTracks |
|---|---|---|
| voice conversation | Audio, Data | Audio, Data |
| voice conversation with visual input | Audio, Video, Data | Audio, Data |
4. Establish Connection
Fill the credentials and track configuration into AoqConnectConfig, then call:
onConnectionStatusChange to return .connected, then proceed to the model initialization process. A return from connect does not mean that the connection has succeeded.
5. Exchange model events and media
Send and parse model events
The client sends events via the Data track. The following is a Swift code snippet for iOS; for other interfaces, see iOS SDK Reference:
onDataMsg. This model identifies event types through the type field. Complete fields, parameters, and termination conditions should follow the target model’s documentation.
Enable media after the model is ready
After the current model, session, or task satisfies its readiness conditions, enable only the configured upstream directions. For speech input, call:
.video only when video input is required.
Transmit audio and video on their media tracks, and model events and text on Data tracks. For example, Omni over AOQ does not require input_audio_buffer.append or input_image_buffer.append to send the same media again. Handle downstream audio with the configured player or external playback logic.
6. End interaction and release resources
When the connection is no longer needed, call the AOQ SDK disconnect and resource-release interfaces from your application:
Sending and Receiving Events via the Data Channel
The client sends and receives model events via the Data track of AOQ SDK. During connection, both publishing and subscribing to the Data track are required; when sending, serialize the model event into AoqDataMsg and call the platform’s Data send interface; upon receipt, parse the message in the onDataMsg callback. See Platform-Specific SDK for detailed interfaces.
The names, fields, parameters, and timing of sent events follow the target model’s Client Event Definitions; the parsing method, state changes, results, and error handling after receiving messages adhere to the model’s Server Event Definitions. Initialization events are sent as needed according to the model’s requirements, while subsequent text and control events are sent based on business needs.
| Model or application | Client events (send) | Server events (receive) |
|---|---|---|
| Qwen-Omni-Realtime | Client Events | Server Events |
-
Use the
typefield of events to identify model events, and handle associated identifiers according to this model’s event reference. - Operations such as initialization confirmation, input submission, response cancellation, and task completion—required by the model—are all handled within the Data channel according to the corresponding protocol; these constitute the model’s interaction semantics and are distinct from disconnecting the transport connection.
- Before ending, complete input submission and result reception as described in this model’s event documentation; if full playback is required, release the connection only after processing any locally queued audio.
- Audio and video data are transmitted via media tracks. When referencing the model event documentation, media encapsulation and available interaction modes still follow the access methods outlined here and do not directly adopt WebSocket’s media transmission approach.
Advanced features
| Capability | Use case | Documentation |
|---|---|---|
| Control audio and video transmission separately | Wait for model readiness, pause upstream media, or enable video as needed | Media transmission management |
| Audio devices and processing | Encoding, decoding, speakers, file mixing, and audio frame callbacks | Common audio features |
| External audio input | Use an application-provided audio source | Custom audio capture |
| External audio output | Process or play output audio in your application | Custom audio playback |
| Video capture and input | Use a camera or application-provided images | Common video features, Custom video input |
Session Configuration Example
qwen3.8-omni-flash-realtime. Audio and video are transmitted via media tracks; the media send events of WebSocket are not used.AOQ SDK downloads
The version update notes below describe the general capabilities of AOQ SDK; the scope of media support for this model is detailed in the Media Orientation for This Model above.
| Version | Platform | Download | Release Date | Release Notes |
|---|---|---|---|---|
| v1.4.0 | Android | AoqClientSdk-release.aarlibPluginOpus.zip | 2026-10-27 |
|
| iOS | AoqClientSdk.framework.zipAoqScreenShare.framework.zipPluginOpus.framework.zip | |||
| HarmonyOS | AoqClientSdk.harlibPluginOpus.zip | |||
| Windows | AoqClientSdk-windows-x64.zip | |||
| macOS | AoqClientSdk.framework.zipPluginOpus.framework.zip | |||
| Electron | aoq-electron-sdk-1.4.0-59463182.tgzAlternatively, install through npm:npm install aoq-electron-sdk@1.4.0-59463182 | |||
| Linux | x86_64: AoqClientSdk.ziparm64: AoqClientSdk.zip | |||
| v1.3.0 | Android | 2026-09-18 |
| |
| iOS | ||||
| HarmonyOS | ||||
| Windows | AoqClientSdk-windows-x64.zip | |||
| macOS | ||||
| Electron | Or install with npm: npm install aoq-electron-sdk@1.3.0-58586345 | |||
| Linux | x86_64: AoqClientSdk.zip arm64: AoqClientSdk.zip | |||
| v1.2.2 | iOS | AoqClientSdk.framework.zip | 2026-09-09 |
|
| v1.2.0 | Android | 2026-08-17 |
| |
| iOS | ||||
| HarmonyOS | ||||
| Windows | ||||
| macOS | ||||
| Electron | Package: aoq-electron-sdk-1.2.0-57407598.tgz Alternatively, install it by using npm: npm install aoq-electron-sdk@1.2.0-57407598 | |||
| Linux | x86_64: AoqClientSdk.zip arm64: AoqClientSdk.zip | |||
| v1.1.0 | Android | 2026-07-31 |
| |
| iOS | ||||
| HarmonyOS | ||||
| Linux | AoqClientSdk.zip | |||
| v1.0.1 | Android | 2026-07-09 |
| |
| iOS | ||||
| HarmonyOS |