Skip to main content
Sambert

Sambert speech synthesis HarmonyOS SDK

This guide explains how to use the Sambert speech synthesis HarmonyOS SDK to convert text into high-quality, expressive speech.

User guide: For model introductions and selection recommendations, see Speech synthesis - Sambert. Online experience: Not supported.
Alibaba Cloud Model Studio has introduced a workspace-specific domain for the China (Beijing) region. The domain provides superior performance and higher stability for inference requests. We recommend migrating from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.
Replace {WorkspaceId} with your actual workspace ID. The existing domain remains available.

NativeNui

This SDK is based on the NativeNui architecture and uses callbacks to process speech synthesis events. Architecture characteristics:

Procedure

  1. tts_initialize - Initialize the SDK and configure the callback interface and connection parameters.
  2. setParamTts - Configure speech synthesis effect parameters such as the model, voice, and volume.
  3. startTts - Start a speech synthesis task.
  4. onTtsDataCallback - Receive audio data.
  5. tts_release - Release SDK resources.

Sambert methods

tts_initialize

Initializes a speech synthesis SDK instance. Create an instance by using new NativeNui(Constants.ModeType.MODE_TTS). Each instance corresponds to one speech synthesis channel. Do not initialize the same instance again before you call tts_release. To process multiple tasks concurrently, create multiple instances. This interface blocks the calling thread. Call it from a non-UI thread. Method signature: 
public tts_initialize(callback: NuiTtsSdkListener,
                      ticket: string,
                      level: number,
                      save_log: boolean): number
Parameter descriptions:
ParameterTypeDescription
callbackNuiTtsSdkListenerAn implementation of the event and data callback interface.
ticketstringA JSON string that contains authentication, connection, and debugging parameters. See the ticket parameter descriptions below.
levelnumberControls the SDK log level. Valid values are defined by the Constants.LogLevel enumeration.
save_logbooleanSpecifies whether to save local logs. If this parameter is true, use debug_path in the ticket parameters to specify a path. You can also use max_log_file_size to set the file size.
Return value: An error code. ticket JSON example: 
{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
    "apikey": "sk-****",
    "device_id": "my_device_id",
    "mode_type": "2"
}
ticket parameter descriptions:
ParameterTypeRequiredDescription
urlstringYesThe endpoint. This is fixed at wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference. Replace {WorkspaceId} with your actual workspace ID.
apikeystringYesThe API key. We recommend using a more secure temporary API key with a short validity period to reduce the risk of leaking a long-term key.
mode_typestringYesThe mode type. This must be set to the string "2", which indicates online speech synthesis mode and corresponds to Constants.TtsModeTypeCloud.
device_idstringYesA unique string that identifies the end user. You can set it to an in-app user ID or a unique device identifier generated by the client. This ID is mainly used for log tracking and troubleshooting.
debug_pathstringNoThe storage path for log files. This parameter takes effect only if you set save_log to true when you call tts_initialize. In this case, you must specify a log file path. Otherwise, an error occurs. A maximum of two log files are retained locally.
max_log_file_sizenumberNoSets the maximum size of a log file in bytes. This parameter takes effect only if you set save_log to true when you call tts_initialize. Default value: 104857600 (100 * 1024 * 1024 bytes, or 100 MiB).

setParamTts

Sets speech synthesis effect parameters as key-value pairs. Call this method before startTts. Method signature: 
public setParamTts(param: string, value: string): number
Parameter descriptions:
ParameterTypeDescription
paramstringThe parameter name.
valuestringThe parameter value.
Return value: An error code. Available parameters:
ParameterTypeRequiredDescription
modelstringYesThe model name, such as sambert-zhinan-v1.
formatstringNoThe audio encoding format. Valid values: - pcm - wav - mp3 (default).
volumestringNoThe volume. Default value: 50. Valid range: [0, 100].
sample_ratestringNoThe audio sample rate in Hz. Valid values: 8000, 16000, 22050, 24000, and 48000. The default sample rate for most Sambert voice models is 48000. Configure the player to use the corresponding sample rate.
ratestringNoThe speech rate. Default value: 1.0. Valid range: [0.5, 2.0].
pitchstringNoThe pitch. Default value: 1.0. Valid range: [0.5, 2.0].
word_timestamp_enabledstringNoSpecifies whether to enable word-level timestamps. Default value: false. This parameter applies to all Sambert models.
phoneme_timestamp_enabledstringNoSpecifies whether to enable phoneme-level timestamps. Default value: false. Enable word_timestamp_enabled first.
enable_audio_decoderstringNoSpecifies whether to enable the built-in audio decoder. Default value: 0. Valid values: - 1: enabled. When format is mp3, set this parameter to "1" to enable the built-in SDK decoder. onTtsDataCallback then returns decoded PCM data. - 0: disabled.
enable_callback_volstringNoSpecifies whether to enable the volume callback. Set it to "1" to enable onTtsVolCallback.
apikeystringNoRefreshes a temporary API key during runtime. Before a synthesis task starts, inject the latest temporary key by using setParamTts('apikey', ...).

getparamTts

Obtains a parameter value. This method is mainly used for troubleshooting. Method signature: 
public getparamTts(param: string): string
Parameter descriptions:
ParameterTypeDescription
paramstringThe parameter name. Currently, only "error_msg" is supported.
Return value: The parameter value.

startTts

Starts a speech synthesis task. The synthesis result is returned through callbacks. Method signature: 
public startTts(priority: string, taskid: string, text: string): number
Parameter descriptions:
ParameterTypeDescription
prioritystringThe task priority. Set this parameter to 1.
taskidstringThe task ID. If an empty string '' is passed, the SDK automatically generates an ID.
textstringThe text to synthesize.
Return value: An error code.

pauseTts

Pauses the current speech synthesis task. After the task is paused, call resumeTts to resume it or cancelTts to cancel it. The SDK cannot start a new synthesis task while a task is paused. Note: This operation only pauses data retrieval from the server. Audio data already buffered in the player continues to play. Method signature: 
public pauseTts(): number
Return value: An error code.

resumeTts

Resumes a paused speech synthesis task. Method signature: 
public resumeTts(): number
Return value: An error code.

cancelTts

Cancels a synthesis task. Note: This operation only cancels data retrieval from the server. Audio data already buffered in the player continues to play. Method signature: 
public cancelTts(taskid: string): number
Parameter descriptions:
ParameterTypeDescription
taskidstringThe ID of the task to cancel. If an empty string '' is passed, all paused or active synthesis tasks are canceled.
Return value: An error code.

tts_release

Releases all internal SDK resources and forcibly terminates all active synthesis tasks. After this method is called, the SDK instance becomes unavailable. To use it again, call tts_initialize to reinitialize it. Method signature: 
public tts_release(): number
Return value: An error code.

NuiTtsSdkListener

The Sambert speech synthesis callback interface receives synthesis events and audio data. In the HarmonyOS SDK, callbacks are defined as ArkTS arrow functions.

onTtsEventCallback

Listens for speech synthesis task start, end, cancel, pause, resume, and error events. Method signature: 
onTtsEventCallback: (event: NuiSdkTtsEvent, taskid: string, ret_code: number) => void;
Parameter descriptions:
ParameterTypeDescription
eventNuiSdkTtsEventThe callback event.
taskidstringThe speech synthesis task ID.
ret_codenumberThe error code. This parameter is valid only for the TTS_EVENT_ERROR event.

onTtsDataCallback

During synthesis, the SDK continuously triggers this callback. Obtain audio data from the callback. Method signature: 
onTtsDataCallback: (info: string, info_len: number, buffer: ArrayBuffer | null) => void;
Parameter descriptions:
ParameterTypeDescription
infostringA JSON-formatted timestamp result. This parameter takes effect when word_timestamp_enabled is set to "1".
info_lennumberThe data length of the info field. You can ignore this parameter.
bufferArrayBuffer | nullThe audio data for the current segment. This parameter may be null. Check for null in the callback.
The underlying implementation may reuse buffer. To cache it, make a copy first, such as by using new Uint8Array(buffer.slice(0)).

onTtsVolCallback

After enable_callback_vol is enabled, this callback returns the volume of the synthesis data just received by the SDK. This is not the volume currently being played. Method signature: 
onTtsVolCallback: (vol: number) => void;
Parameter descriptions:
ParameterTypeDescription
volnumberThe volume of the synthesis data.

NuiSdkTtsEvent

The Sambert speech synthesis event type enumeration.
EventDescription
TTS_EVENT_STARTThe synthesis task starts. Audio data is about to be returned.
TTS_EVENT_ENDThe synthesis task ends normally. All audio data has been returned through the callback.
TTS_EVENT_CANCELThe synthesis task is canceled.
TTS_EVENT_PAUSEThe synthesis task is paused.
TTS_EVENT_RESUMEThe synthesis task is resumed.
TTS_EVENT_ERRORAn error occurs during synthesis. Call getparamTts("error_msg") to obtain details. { "header": { "task_id": "xxxxxxxxx", "event": "task-failed", "error_code": "InvalidParameter", "error_message": "Please ensure input text is valid.", "attributes": {} }, "payload": {} }
The TTS_EVENT_END event indicates that TTS synthesis is complete and all audio data has been returned through callbacks. It does not indicate that the player has finished playing all audio data.

Auxiliary types

Constants.LogLevel

The enumeration values for the level parameter are as follows:
ValueDescription
LOG_LEVEL_VERBOSEThe most detailed logs.
LOG_LEVEL_DEBUGDebug logs.
LOG_LEVEL_INFOInformational logs (default).
LOG_LEVEL_WARNINGWarning logs.
LOG_LEVEL_ERRORError logs.
LOG_LEVEL_NONEDisables logging.

Sample code

  1. Obtain an API key: Obtain and configure an API key. For security, we recommend configuring the API key as an environment variable.
    To grant temporary access to third-party applications or users, or to strictly control high-risk operations such as accessing or deleting sensitive data, use a temporary API key. A temporary API key is valid for 60 seconds by default. Obtain a new one after it expires.
  2. Download the SDK and run the sample code:
    • Download the latest SDK package.
    • Extract the TAR package. Obtain the HAR-format SDK from the neonui directory and add it to your project dependencies. For C++ integration, obtain the dynamic libraries and header files from native/libs and native/include in the TAR package.
    • Open the project in DevEco Studio. The sample code is located in DashSambertTtsPage.ets. Replace the API key to try the feature.

Invocation steps

  1. Initialize the SDK: Call tts_initialize and pass the NuiTtsSdkListener callback and ticket parameters.
  2. Configure parameters based on your business requirements: Use setParamTts to configure speech synthesis effect parameters such as the model, format, sample rate, voice, and volume. We recommend configuring them immediately after initialization succeeds.
  3. Call startTts to start speech synthesis.
  4. Obtain audio data from the onTtsDataCallback callback. We recommend streaming playback as described in Audio playback below. To save the audio locally, append the audio data to the same file until synthesis is complete.
  5. After the task ends, call tts_release to release SDK resources.

Audio playback

HarmonyOS uses AudioRenderer from @kit.AudioKit to play synthesized audio. The default sample rate of Sambert synthesized audio is 48 kHz, so configure the player sample rate to 48000. The product sample encapsulates the logic in the AudioPlayer.ets utility class. Specify the sample rate in the constructor: 
// Play Sambert audio at the default sample rate of 48000. The default for AudioPlayer is 16000.
this.mAudioPlayer = new AudioPlayer(this, 48000);
The player uses the writeData callback to retrieve audio data from a queue and returns AudioDataCallbackResult.VALID or AudioDataCallbackResult.INVALID. When onTtsEventCallback receives TTS_EVENT_END, the SDK has completed synthesis and returned all data through the callbacks. Mark the playback queue as complete so that the player automatically stops after playing the remaining data.
MP3 playback: AudioRenderer supports only PCM playback. When format is set to mp3, also set enable_audio_decoder to "1". The built-in SDK decoder decodes MP3 into PCM and returns it through onTtsDataCallback. mEncodeType is used only as the file name extension of the generated audio file and does not affect the callback data type.
API Usage
Text Generation
Image Generation
  • FAQ
Video Generation
World models
Audio
  • Audio generation
Real-time Multimodal
Realtime API
Text Embedding
More Models
Toolkit/Framework
TokenPlan
Model Production
File Management
Billing
Model Management
More