Paraformer real-time speech recognition HarmonyOS SDK

​

Quick start

1. Obtain an API key: Obtain and configure an API key. For security, we recommend configuring the API key as an environment variable.
   To grant temporary access to third-party applications or users, or to strictly control high-risk operations such as accessing or deleting sensitive data, use a temporary API key. A temporary API key is valid for 60 seconds by default. Obtain a new one after it expires.
2. Download the SDK and run the sample code:
   • Download the latest SDK package.
   • Extract the TAR package. Obtain the HAR-format SDK from the neonui directory and add it to your project dependencies. For C++ integration, obtain the dynamic libraries and header files from native/libs and native/include in the TAR package.
   • Open the project in DevEco Studio. The sample code is located in DashParaformerSpeechTranscriberPage.ets. Replace the API key to try the feature.

​

Invocation steps

1. Initialize the SDK.
2. Configure parameters based on your business requirements: Use the parameters parameter of initialize to set connection and control parameters, and use setParams to set speech recognition effect parameters.
3. Call startDialog to start recognition.
4. In the onNuiAudioStateChanged callback, start the recording device based on the audio state.
5. In the onNuiNeedAudioData callback, continuously provide recorded audio data.
6. In the onNuiEventCallback callback, listen for events and obtain speech recognition results.
7. Call stopDialog to stop recognition, and listen for the EVENT_TRANSCRIBER_COMPLETE event to confirm that recognition has ended.
8. When speech recognition is no longer needed, call release to release SDK resources.

​

Request parameters

​

Connection and control parameters

{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
    "apikey": "st-****",
    "device_id": "my_device_id",
    "service_mode": "1"
}

• Parameter descriptions

​

Speech recognition effect parameters

{
    "service_type": 4,
    "nls_config": {
        "model": "paraformer-realtime-v2",
        "sr_format": "pcm",
        "sample_rate": "16000"
    }
}

• Parameter descriptions

​

Key interfaces

​

NativeNui

​

initialize

• Method signature

public initialize(callback: INativeNuiCallback,
                  parameters: string,
                  level: number,
                  save_log: boolean = false): number

• Parameter descriptions

• Return value

​

setParams

• Method signature

public setParams(params: string): number

• Parameter descriptions

• Return value

​

startDialog

• Method signature

public startDialog(vad_mode: Constants.VadMode, dialog_params: string): number

• Parameter descriptions

• Return value

​

stopDialog

• Method signature

public stopDialog(): number

• Return value

​

cancelDialog

• Method signature

public cancelDialog(): number

• Return value

​

release

• Method signature

public release(): number

• Return value

​

GetVersion

• Method signature

public GetVersion(): string

• Return value

​

INativeNuiCallback

​

onNuiEventCallback

• Method signature

onNuiEventCallback: (event: Constants.NuiEvent, resultCode: number, arg2: number,
                    kwsResult: KwsResult, asrResult: AsrResult) => void;

• Parameter descriptions

​

onNuiAudioStateChanged

• Method signature

onNuiAudioStateChanged: (state: Constants.AudioState) => void

• AudioState descriptions

​

onNuiAudioRMSChanged

• Method signature

onNuiAudioRMSChanged: (val: number) => number

• Parameter descriptions

​

onNuiNeedAudioData

• Method signature

onNuiNeedAudioData: (buffer: ArrayBuffer) => number

• Parameter descriptions

• Return value

​

onNuiLogTrackCallback

onNuiLogTrackCallback: (level: Constants.LogLevel, log: string) => void

​

Constants.NuiEvent

​

Auxiliary types

​

Constants.LogLevel

​

Audio device management

• Creation: audio.createAudioCapturer(capturerOptions) asynchronously creates the capturer with a fixed sample rate of 16 kHz, 16-bit depth, and one channel (SAMPLE_RATE_16000/CHANNEL_1/SAMPLE_FORMAT_S16LE/ENCODING_TYPE_RAW).
• Data event: capturer.on('readData', (buffer: ArrayBuffer) => void) continuously obtains recorded audio data. Buffer the data in a queue so that the onNuiNeedAudioData callback can retrieve it as needed.
• State event: capturer.on('stateChange', (state: audio.AudioState) => void). STATE_RUNNING indicates that recording has started, and STATE_STOPPED indicates that recording has stopped.
• Control: start() starts, stop() stops, and release() releases the capturer.

​

Permission declaration

{
  "requestPermissions": [
    { "name": "ohos.permission.MICROPHONE" }
  ]
}