The parameters and interfaces of the Paraformer real-time speech recognition Python SDK.
Prerequisites
Model list
| paraformer-realtime-v2 | paraformer-realtime-8k-v2 | |
|---|---|---|
| Scenarios | Live streaming, meetings, and similar scenarios | Recognition of 8 kHz audio in scenarios such as telephone customer service and voicemail |
| Sample rate | Any | 8kHz |
| Language | Chinese (including Mandarin and various dialects), English, Japanese, Korean, German, French, RussianSupported Chinese dialects: Shanghainese, Wu, Minnan, Northeastern, Gansu, Guizhou, Henan, Hubei, Hunan, Jiangxi, Ningxia, Shanxi, Shaanxi, Shandong, Sichuan, Tianjin, Yunnan, Cantonese | Chinese |
| Punctuation prediction | ✅ Supported by default. No configuration is required. | ✅ Supported by default. No configuration is required. |
| Inverse Text Normalization (ITN) | ✅ Supported by default. No configuration is required. | ✅ Supported by default. No configuration is required. |
| Custom vocabulary | ✅ See Customize hotwords | ✅ See Customize hotwords |
| Specify recognition language | ✅ Specify the language using the language_hints parameter. | ❌ |
| Emotion recognition | ❌ |
✅ (Click to view usage) Emotion recognition has the following constraints:
emo_tag and emo_confidence fields of the single-sentence information (Sentence), respectively. |
Getting started
The Recognition class provides methods for non-streaming and bidirectional streaming calls. You can select the appropriate method based on your requirements:
- Non-streaming call: Recognizes a local file and returns the complete result at once. This is suitable for processing pre-recorded audio.
- Bidirectional streaming call: Recognizes an audio stream and outputs the results in real time. The audio stream can come from an external device, such as a microphone, or be read from a local file. This is suitable for scenarios that require immediate feedback.
Non-streaming call
This method submits a real-time speech-to-text task for a local file. The process is blocked until the complete transcription result is returned.
call method to perform recognition or translation and obtain the RecognitionResult.
Click to view the complete example
Click to view the complete example
result.get_sentence() returns a list of sentences (List[Dict]). Each element contains fields such as text (recognized text), begin_time / end_time (timestamps), and words (per-word timestamps). Iterate through the list and use sentence['text'] to extract plain text.This differs from a streaming callback, where result.get_sentence() returns a single sentence (Dict[str, Any]). For details, see Sentence.Bidirectional streaming call
This method submits a real-time speech-to-text task and returns real-time recognition results through a callback interface.
-
Start streaming speech recognition
Instantiate the Recognition class, bind the request parameters and the callback interface (RecognitionCallback), and call the
startmethod to start streaming speech recognition. -
Streaming
Repeatedly call the Recognition class's
send_audio_framemethod to send the binary audio stream from a local file or a device (such as a microphone) to the server in segments. As audio data is sent, the server uses the RecognitionCallback callback interface'son_eventmethod to return the recognition results to the client in real time. We recommend that the duration of each audio segment sent is about 100 milliseconds, and the data size is between 1 KB and 16 KB. -
End processing
Call the
stopmethod of the Recognition class to stop speech recognition. This method blocks the current thread until theon_completeoron_errorcallback of the callback interface (RecognitionCallback) is triggered.
Click to view the complete example
Click to view the complete example
Concurrent calls
In Python, because of the Global Interpreter Lock (GIL), only one thread can execute Python code at a time (although some performance-oriented libraries may remove this limitation). If you want to better utilize the computing resources of a multi-core computer, we recommend that you use multiprocessing or concurrent.futures.ProcessPoolExecutor. Multi-threading can significantly increase SDK call latency under high concurrency.
Request parameters
Request parameters are set in the constructor (init) of the Recognition class.
Parameter | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | - | Yes | The model used for real-time speech recognition. For more information, see Model List. |
sample_rate | int | - | Yes | Set the sample rate (in Hz) of the audio to be recognized. Varies by model:
|
format | str | - | Yes | Set the audio format to be recognized. Supported audio formats: pcm, wav, mp3, opus, speex, aac, amr. opus/speex: Must use Ogg encapsulation. wav: Must be PCM encoded. amr: Only AMR-NB type is supported. |
vocabulary_id | str | - | No | Set the hot word ID. If not set, hot words will not take effect. Use this field to set the hot word ID for v2 and later models. In the current speech recognition session, the hot word information corresponding to this hot word ID will be applied. For detailed usage, see Custom hotwords. |
disfluency_removal_enabled | bool | False | No | Set whether to filter filler words:
|
language_hints | list[str] | ["zh", "en"] | No | Set the language codes for recognition. If you cannot determine the language in advance, you can leave this unset and the model will automatically detect the language. Currently supported language codes:
This parameter applies only to multilingual models. For more information, see Model list. |
semantic_punctuation_enabled | bool | False | No | Set whether to enable semantic segmentation. Disabled by default.
Semantic segmentation provides higher accuracy and is suitable for meeting transcription scenarios. VAD (Voice Activity Detection) segmentation has lower latency and is suitable for interactive scenarios. By adjusting the This parameter only takes effect when the model is v2 or later. |
max_sentence_silence | int | 800 | No | Set the silence duration threshold (in ms) for VAD (Voice Activity Detection) segmentation. When the silence duration after a speech segment exceeds this threshold, the system determines that the sentence has ended. The parameter range is 200 ms to 6000 ms, with a default value of 800 ms. This parameter only takes effect when the |
multi_threshold_mode_enabled | bool | False | No | When this switch is enabled (true), it prevents VAD segmentation from cutting sentences that are too long. Disabled by default. This parameter only takes effect when the |
punctuation_prediction_enabled | bool | True | No | Set whether to automatically add punctuation in the recognition results:
This parameter only takes effect when the model is v2 or later. |
heartbeat | bool | False | No | When you need to maintain a long connection with the server, use this switch to control the behavior:
This parameter only takes effect when the model is v2 or later. When using this field, the SDK version must be 1.23.1 or later. |
inverse_text_normalization_enabled | bool | True | No | Set whether to enable ITN (Inverse Text Normalization). Enabled by default (true). When enabled, Chinese numerals are converted to Arabic numerals. This parameter only takes effect when the model is v2 or later. |
callback | RecognitionCallback | - | No |
Key interfaces
Recognition class
The Recognition class is imported using from dashscope.audio.asr import *.
| Member method | Method signature | Description |
|---|---|---|
| call | A non-streaming call that uses a local file. This method blocks the current thread until the entire audio file is read. The file must have read permissions.The recognition result is returned as a RecognitionResult type. | |
| start | Starts speech recognition.This is a callback-based streaming real-time recognition method that does not block the current thread. It must be used with send_audio_frame and stop. | |
| send_audio_frame | Pushes an audio stream. The audio stream pushed each time should not be too large or too small. We recommend that each audio packet has a duration of about 100 ms and a size between 1 KB and 16 KB.You can obtain the recognition results through the on_event method of the callback interface (RecognitionCallback). | |
| stop | Stops speech recognition. This method blocks until the service has recognized all received audio and the task is complete. | |
| get_last_request_id | Gets the request_id. This can be used after the constructor is called (the object is created). | |
| get_first_package_delay | Gets the first packet delay, which is the latency from sending the first audio packet to receiving the first recognition result packet. Use this after the task is completed. | |
| get_last_package_delay | Obtains the last packet delay, which is the time taken from sending the stop instruction to receiving the last recognition result packet. Use this after the task is completed. |
Callback interface (RecognitionCallback)
During a bidirectional streaming call, the server uses callbacks to return key process information and data to the client. You must implement a callback method to process the returned information and data.
Click to view the example
Click to view the example
| Method | Parameter | Return value | Description |
|---|---|---|---|
| None | None | This method is called immediately after a connection is established with the server. | |
result: RecognitionResult | None | This method is called when the service sends a response. | |
| None | None | This method is called after all recognition results have been returned. | |
result: Recognition result | None | This method is called when an exception occurs. | |
| None | None | This method is called after the service has closed the connection. |
Response results
Recognition result (RecognitionResult)
RecognitionResult represents the recognition result of either a single real-time recognition in a bidirectional streaming call or a non-streaming call.
| Member method | Method signature | Description |
|---|---|---|
| get_sentence | Gets the currently recognized sentence and timestamp information. In a callback, a single sentence is returned, so this method returns a Dict[str, Any] type.For more information, see Sentence. | |
| get_request_id | Gets the request_id of the request. | |
| is_sentence_end | Determines whether the given sentence has ended. |
Sentence (Sentence)
The members of the Sentence class are as follows:
Parameter | Type | Description |
|---|---|---|
begin_time | int | The start time of the sentence, in ms. |
end_time | int | The end time of the sentence, in ms. |
text | str | The recognized text. |
words | A list of Word timestamp information (Word) | Word timestamp information. |
emo_tag | str | The emotion of the current sentence:
Emotion recognition has the following constraints:
|
emo_confidence | float | The confidence level of the recognized emotion for the current sentence. The value ranges from 0.0 to 1.0. A larger value indicates a higher confidence level. Emotion recognition has the following constraints:
|
Word timestamp information (Word)
The members of the Word class are as follows:
Parameter | Type | Description |
|---|---|---|
begin_time | int | The start time of the word, in ms. |
end_time | int | The end time of the word, in ms. |
text | str | The word. |
punctuation | str | The punctuation. |
Error codes
If you encounter errors, see Error codes for troubleshooting.
If the issue persists, join the developer community to report your issue and provide the Request ID for further investigation.
More examples
For more examples, see GitHub.
FAQ
Features
Q: How to maintain a long connection with the server during prolonged silence?
Set the request parameter heartbeat to true and continuously send silent audio to the server.
Silent audio refers to audio files or data streams that contain no sound signal. Silent audio can be generated through various methods, such as using audio editing software like Audacity or Adobe Audition, or through command-line tools like FFmpeg.
Q: How to convert audio to a supported format?
You can use the FFmpeg tool. For more usage, refer to the FFmpeg official website.
Q: Does it support viewing the time range for each sentence?
Yes. The speech recognition results include the start and end timestamps for each sentence, which can be used to determine the time range of each sentence.Q: How do I recognize a local file (recorded audio file)?
There are two ways to recognize a local file:
-
Directly pass the local file path: This method returns the complete recognition result after the file is fully processed. It is not suitable for scenarios that require immediate feedback.
Pass the file path to the
callmethod of the Recognition class to directly recognize the audio file. For more information, see Non-streaming call. -
Convert the local file into a binary stream for recognition: This method returns recognition results as a stream while the file is being processed. It is suitable for scenarios that require immediate feedback.
You can use the
send_audio_framemethod of the Recognition class to send a binary stream to the server for recognition. For more information, see bidirectional streaming call.
Troubleshooting
Q: What causes the failure to recognize speech (no recognition results)?
-
Check whether the audio format (
format) and sample rate (sampleRate/sample_rate) in the request parameters are correctly set and comply with parameter constraints. The following are common error examples:- The audio file extension is .wav, but the actual format is MP3, and the request parameter
formatis set to mp3 (incorrect parameter setting). - The audio sample rate is 3600 Hz, but the request parameter
sampleRate/sample_rateis set to 48000 (incorrect parameter setting).
- The audio file extension is .wav, but the actual format is MP3, and the request parameter
-
When using the
paraformer-realtime-v2model, check whether the language set inlanguage_hintsmatches the actual language of the audio. For example: The audio is actually in Chinese, butlanguage_hintsis set toen(English). - If all the above checks pass, you can use custom hot words to improve recognition accuracy for specific words.