Use the speech Node.js SDK to configure Qwen-Audio-Realtime sessions, exchange audio, and handle transcription and function calling.
Install the SDK
Requires Node.js 18 or later. Download the SDK 0.1.0 package, then run this command in your project:
Quick start
Obtain an API key in the console and set it as the DASHSCOPE_API_KEY environment variable.
Download the sample audio as input.mp3 and convert it to 16 kHz, 16-bit, mono PCM:
example.mjs and run node example.mjs.
API reference
Constructor options
Constructor options for RealtimeClient:
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | No | API key. Defaults to DASHSCOPE_API_KEY. |
workspaceId | string | No | Workspace ID. Defaults to DASHSCOPE_WORKSPACE_ID. |
baseUrl | string | No | Set the service hostname to dashscope-intl.aliyuncs.com (Singapore). |
model | string | Yes | Model ID. |
voice | string | No | Voice ID. Uses the service default if omitted. |
instructions | string | No | System instructions for role, style, and other behavior. |
turnDetection | TurnDetection | null | No | Default: { type: "server_vad" }. null enables manual submission. Also supports smart_turn and smart_turn_v2. |
modalities | Array<'text' | 'audio'> | No | Output modalities. Default: ["text", "audio"]. |
maxHistoryTurns | number | No | History turns: [1,50]. Service default: 20. |
tools | ToolDefinition[] | No | Tool definitions. The application executes tools and validates their arguments. |
inputTranscription | { model?: string; language?: string } | null | No | Input transcription settings, such as { model: "fun-asr", language: "zh" }; null disables transcription. |
sessionExtras | SessionConfig | No | Additional session settings, such as speech_rate and enable_search. Override matching top-level settings; see client events for the fields. |
connectTimeoutMs | number | No | Connection timeout in ms. Default: 10000. |
Main methods
| Method | Return value | Description |
|---|---|---|
connect() | Promise<void> | Connect and send session settings. sessionUpdated confirms they took effect. |
updateSession(session: SessionConfig) | void | Update session settings; confirm with sessionUpdated. |
appendAudio(pcm: Buffer | Uint8Array) | void | Send 16 kHz, 16-bit, mono PCM audio. |
commit() | void | Commit the audio buffer in manual mode. |
clearAudioBuffer() | void | Clear the uncommitted audio buffer. |
createResponse(response?: Record<string, unknown>) | void | Request a model response. |
cancelResponse() | void | Cancel the current response and keep the session. |
createConversationItem(item: ConversationItem, previousItemId?: string) | void | Add conversation history or tool results. previousItemId sets the insertion position. |
submitFunctionCallOutput(callId: string, output: string, autoCreateResponse?: boolean) | void | Submit a tool-result string. autoCreateResponse defaults to true and requests the next response. |
deleteConversationItem(itemId: string) | void | Delete a conversation item. |
retrieveConversationItem(itemId: string) | void | Retrieve an item; receive it through event. |
sendEvent(event: ClientEvent) | void | Send a raw client event. |
close(code?: number, reason?: string) | void | Close the session and WebSocket connection. |
Callback events
Register callbacks with on(event, listener). See server events for the full event fields.
| Event | Payload | Description |
|---|---|---|
open | void | Connection established and initial configuration sent. |
sessionCreated / sessionUpdated | ServerEvent | Session creation or configuration confirmation. |
audio | Buffer | Binary audio chunk; may fire multiple times. |
audioTranscriptDelta | { delta: string } | Incremental output-audio transcript. |
outputTranscript | { transcript: string } | Complete output-audio transcript. |
textDelta / textDone | { delta: string } / { text: string } | Incremental or complete text in text-only modality. |
inputTranscriptDelta | { text: string; stash: string } | text is confirmed text and stash is tentative text. Replace the current display with their combination rather than appending it. |
inputTranscript | { transcript: string } | Final input-audio transcript. |
speechStarted / speechStopped | ServerEvent | User speech started or stopped. |
responseCreated / responseDone | ServerEvent | Response started or ended. Check whether responseDone.response.status is completed, canceled, or failed. |
functionCall | { callId: string; name: string; arguments: string } | Complete function-call arguments. The application parses and validates the arguments JSON string. |
ambientTranscript / ambientTranscriptCompleted | { text: string; stash: string } / { transcript: string } | Current or final ambient-audio transcript, excluded from conversation context. |
event | ServerEvent | Raw service JSON message. |
error | DashScopeError | Connection or service error. |
close | { code: number; reason: string } | Underlying connection-close notification. |