Skip to main content

Qwen-Audio-Realtime Node.js SDK

Use the speech Node.js SDK to configure Qwen-Audio-Realtime sessions, exchange audio, and handle transcription and function calling.

Install the SDK

Requires Node.js 18 or later. Download the SDK 0.1.0 package, then run this command in your project:
npm install ./dashscope-sdk-nodejs-0.1.0.tgz

Quick start

Obtain an API key in the console and set it as the DASHSCOPE_API_KEY environment variable. Download the sample audio as input.mp3 and convert it to 16 kHz, 16-bit, mono PCM:
ffmpeg -i input.mp3 -f s16le -ac 1 -ar 16000 input.pcm
Save the following example as example.mjs and run node example.mjs.
import { readFileSync, writeFileSync } from 'node:fs'
import { setTimeout as sleep } from 'node:timers/promises'
import { RealtimeClient } from 'dashscope-sdk-nodejs/realtime'

const pcm = readFileSync('input.pcm')
const client = new RealtimeClient({
  apiKey: process.env.DASHSCOPE_API_KEY,
  workspaceId: process.env.DASHSCOPE_WORKSPACE_ID,
  baseUrl: 'dashscope-intl.aliyuncs.com',
  model: 'qwen-audio-3.1-realtime-plus',
  voice: 'longanqian_v3.1',
  turnDetection: null,
  inputTranscription: { language: 'zh' },
})
const audio = []
function fail(err) {
  console.error(err.code, err.message)
  process.exitCode = 1
  client.close()
}
const timer = setTimeout(() => fail(new Error('Response timed out')), 60000)
client.on('close', () => clearTimeout(timer))
client.on('error', fail)
client.on('audio', chunk => audio.push(chunk))
client.on('inputTranscript', ({ transcript }) => console.log('User:', transcript))
client.on('outputTranscript', ({ transcript }) => console.log('Assistant:', transcript))
client.once('sessionUpdated', () => {
  ;(async () => {
    for (let offset = 0; offset < pcm.length; offset += 3200) {
      client.appendAudio(pcm.subarray(offset, offset + 3200))
      await sleep(100)
    }
    client.commit()
    client.createResponse()
  })().catch(fail)
})
client.once('responseDone', event => {
  if (event.response?.status !== 'completed') {
    fail(new Error('Response status: ' + event.response?.status))
    return
  }
  writeFileSync('output.pcm', Buffer.concat(audio))
  client.close()
})
try {
  await client.connect()
} catch (err) {
  fail(err)
}

API reference

Constructor options

Constructor options for RealtimeClient:
ParameterTypeRequiredDescription
apiKeystringNoAPI key. Defaults to DASHSCOPE_API_KEY.
workspaceIdstringNoWorkspace ID. Defaults to DASHSCOPE_WORKSPACE_ID.
baseUrlstringNoSet the service hostname to dashscope-intl.aliyuncs.com (Singapore).
modelstringYesModel ID.
voicestringNoVoice ID. Uses the service default if omitted.
instructionsstringNoSystem instructions for role, style, and other behavior.
turnDetectionTurnDetection | nullNoDefault: { type: "server_vad" }. null enables manual submission. Also supports smart_turn and smart_turn_v2.
modalitiesArray<'text' | 'audio'>NoOutput modalities. Default: ["text", "audio"].
maxHistoryTurnsnumberNoHistory turns: [1,50]. Service default: 20.
toolsToolDefinition[]NoTool definitions. The application executes tools and validates their arguments.
inputTranscription{ model?: string; language?: string } | nullNoInput transcription settings, such as { model: "fun-asr", language: "zh" }; null disables transcription.
sessionExtrasSessionConfigNoAdditional session settings, such as speech_rate and enable_search. Override matching top-level settings; see client events for the fields.
connectTimeoutMsnumberNoConnection timeout in ms. Default: 10000.

Main methods

MethodReturn valueDescription
connect()Promise<void>Connect and send session settings. sessionUpdated confirms they took effect.
updateSession(session: SessionConfig)voidUpdate session settings; confirm with sessionUpdated.
appendAudio(pcm: Buffer | Uint8Array)voidSend 16 kHz, 16-bit, mono PCM audio.
commit()voidCommit the audio buffer in manual mode.
clearAudioBuffer()voidClear the uncommitted audio buffer.
createResponse(response?: Record<string, unknown>)voidRequest a model response.
cancelResponse()voidCancel the current response and keep the session.
createConversationItem(item: ConversationItem, previousItemId?: string)voidAdd conversation history or tool results. previousItemId sets the insertion position.
submitFunctionCallOutput(callId: string, output: string, autoCreateResponse?: boolean)voidSubmit a tool-result string. autoCreateResponse defaults to true and requests the next response.
deleteConversationItem(itemId: string)voidDelete a conversation item.
retrieveConversationItem(itemId: string)voidRetrieve an item; receive it through event.
sendEvent(event: ClientEvent)voidSend a raw client event.
close(code?: number, reason?: string)voidClose the session and WebSocket connection.

Callback events

Register callbacks with on(event, listener). See server events for the full event fields.
EventPayloadDescription
openvoidConnection established and initial configuration sent.
sessionCreated / sessionUpdatedServerEventSession creation or configuration confirmation.
audioBufferBinary audio chunk; may fire multiple times.
audioTranscriptDelta{ delta: string }Incremental output-audio transcript.
outputTranscript{ transcript: string }Complete output-audio transcript.
textDelta / textDone{ delta: string } / { text: string }Incremental or complete text in text-only modality.
inputTranscriptDelta{ text: string; stash: string }text is confirmed text and stash is tentative text. Replace the current display with their combination rather than appending it.
inputTranscript{ transcript: string }Final input-audio transcript.
speechStarted / speechStoppedServerEventUser speech started or stopped.
responseCreated / responseDoneServerEventResponse started or ended. Check whether responseDone.response.status is completed, canceled, or failed.
functionCall{ callId: string; name: string; arguments: string }Complete function-call arguments. The application parses and validates the arguments JSON string.
ambientTranscript / ambientTranscriptCompleted{ text: string; stash: string } / { transcript: string }Current or final ambient-audio transcript, excluded from conversation context.
eventServerEventRaw service JSON message.
errorDashScopeErrorConnection or service error.
close{ code: number; reason: string }Underlying connection-close notification.