The parameters and interfaces of the Paraformer real-time speech recognition Python SDK.
Prerequisites
Model list
| paraformer-realtime-v2 (Recommended) | paraformer-realtime-8k-v2 (Recommended) | |
|---|---|---|
| Scenarios | Live streaming, meetings, and similar scenarios | Recognition of 8 kHz audio in scenarios such as telephone customer service and voicemail |
| Sample rate | Any | 8kHz |
| Language | Chinese (including Mandarin and various dialects), English, Japanese, Korean, German, French, RussianSupported Chinese dialects: Shanghainese, Wu, Minnan, Northeastern, Gansu, Guizhou, Henan, Hubei, Hunan, Jiangxi, Ningxia, Shanxi, Shaanxi, Shandong, Sichuan, Tianjin, Yunnan, Cantonese | Chinese |
| Punctuation prediction | Supported by default. No configuration is required. | Supported by default. No configuration is required. |
| Inverse Text Normalization (ITN) | Supported by default. No configuration is required. | Supported by default. No configuration is required. |
| Custom vocabulary | See Customize hotwords | See Customize hotwords |
| Specify recognition language | Specify the language using the language_hints parameter. | Not supported |
| Emotion recognition | Not supported |
Click to view usage Emotion recognition has the following constraints:
emo_tag and emo_confidence fields of the single-sentence information (Sentence), respectively. |
Quick start
The core resource (client.audio.asr) provides non-streaming and bidirectional streaming call interfaces. You can select the appropriate method based on your requirements:
- Non-streaming call (
recognize_file): Recognizes a local file. The SDK automatically sends the audio in chunks and drives the task to completion. This is suitable for processing pre-recorded audio. - Bidirectional streaming call: Recognizes an audio stream and outputs the results in real time. The audio stream can come from an external device, such as a microphone, or be read from a local file. This is suitable for scenarios that require immediate feedback.
Non-streaming call
This method submits a real-time speech-to-text task for a local file. The process is blocked until the complete transcription result is returned.
recognize_file method of the core resource (client.audio.asr) with the request parameters and a local file path, and iterate the server events to obtain the RecognitionResult.
Click to view the complete example
Click to view the complete example
recognize_file yields server events one by one (Dict[str, Any]): the event type is located in event['header']['event'], and the recognized text is located in event['payload']['output']['sentence'] (sentence_end being true indicates that the sentence has ended; begin_time / end_time are timestamps, words contains per-word timestamps, and other fields are described in Sentence). Heartbeat and empty-text events are still yielded and can be ignored as needed.In a bidirectional streaming session, in addition to events(), you can use results() to iterate only recognition result frames, which return typed RecognitionResult objects with heartbeat frames filtered out. For more information, see the get_sentence method.Bidirectional streaming call
This method submits a real-time speech-to-text task and streams real-time recognition results by iterating events.
-
Start streaming speech recognition
Call the
recognizemethod of the core resource (client.audio.asr) to create a recognition session with the request parameters. Enter thewithcontext (which automatically calls thestartmethod to establish the connection and submit the task) to start streaming speech recognition. -
Streaming
Repeatedly call the session's
send_audio_framemethod to send the binary audio stream read from a local file or a device (such as a microphone) to the server in segments. Sending and receiving must be performed concurrently: the receiver thread iterates server events through event iteration (events()/results()) to return the recognition results to the client in real time. We recommend that the duration of each audio segment sent is about 100 milliseconds, and the data size is between 1 KB and 16 KB. -
End processing
Call the session's
stopmethod to stop speech recognition. This method only notifies the server that the input is complete; the receiver thread must continue to consumeevents()until thetask-finishedevent to obtain the final recognition results and statistics. Errors at the WebSocket layer are raised asDashScopeErrorexceptions, which can be caught with try/except at the sending or receiving points.
Click to view the complete example
Click to view the complete example
Concurrent calls
In Python, because of the Global Interpreter Lock (GIL), only one thread can execute Python code at a time (although some performance-oriented libraries may remove this limitation). If you want to better utilize the computing resources of a multi-core computer, we recommend that you use multiprocessing or concurrent.futures.ProcessPoolExecutor. Multi-threading can significantly increase SDK call latency under high concurrency.
Request parameters
Request parameters are set through the recognize/recognize_file methods of the core resource (client.audio.asr): model, format, sample_rate, vocabulary_id, and disfluency_removal_enabled are named parameters, and the remaining parameters (such as language_hints and semantic_punctuation_enabled) are passed in through the parameters dictionary.
Parameter | Type | Default | Required | Description |
|---|---|---|---|---|
model | str | - | Yes | The model used for real-time speech recognition. For more information, see Model List. |
sample_rate | int | - | Yes | Set the sample rate (in Hz) of the audio to be recognized. Varies by model:
|
format | str | - | Yes | Set the audio format to be recognized. Supported audio formats: pcm, wav, mp3, opus, speex, aac, amr. opus/speex: Must use Ogg encapsulation. wav: Must be PCM encoded. amr: Only AMR-NB type is supported. |
vocabulary_id | str | - | No | Set the hot word ID. If not set, hot words will not take effect. Use this field to set the hot word ID for v2 and later models. In the current speech recognition session, the hot word information corresponding to this hot word ID will be applied. For detailed usage, see Custom hotwords. |
disfluency_removal_enabled | bool | False | No | Set whether to filter filler words:
|
language_hints | list[str] | ["zh", "en"] | No | Set the language codes for recognition. If you cannot determine the language in advance, you can leave this unset and the model will automatically detect the language. Currently supported language codes:
This parameter applies only to multilingual models. For more information, see Model list. |
semantic_punctuation_enabled | bool | False | No | Set whether to enable semantic segmentation. Disabled by default.
Semantic segmentation provides higher accuracy and is suitable for meeting transcription scenarios. VAD (Voice Activity Detection) segmentation has lower latency and is suitable for interactive scenarios. By adjusting the This parameter only takes effect when the model is v2 or later. |
max_sentence_silence | int | 800 | No | Set the silence duration threshold (in ms) for VAD (Voice Activity Detection) segmentation. When the silence duration after a speech segment exceeds this threshold, the system determines that the sentence has ended. The parameter range is 200 ms to 6000 ms, with a default value of 800 ms. This parameter only takes effect when the |
multi_threshold_mode_enabled | bool | False | No | When this switch is enabled (true), it prevents VAD segmentation from cutting sentences that are too long. Disabled by default. This parameter only takes effect when the |
punctuation_prediction_enabled | bool | True | No | Set whether to automatically add punctuation in the recognition results:
This parameter only takes effect when the model is v2 or later. |
heartbeat | bool | False | No | When you need to maintain a long connection with the server, use this switch to control the behavior:
This parameter only takes effect when the model is v2 or later. When using this field, the SDK version must be 1.23.1 or later. |
inverse_text_normalization_enabled | bool | True | No | Set whether to enable ITN (Inverse Text Normalization). Enabled by default (true). When enabled, Chinese numerals are converted to Arabic numerals. This parameter only takes effect when the model is v2 or later. |
API reference
Core resource (client.audio.asr)
Real-time speech recognition is exposed through the client.audio.asr resource: recognize creates a bidirectional streaming recognition session (Recognition), and recognize_file recognizes a local file in one shot.
| Member method | Method signature | Description |
|---|---|---|
| recognize | Creates a bidirectional streaming recognition session. Entering the with context automatically calls start to establish the connection and submit the task.Sending audio and consuming events must be performed concurrently (use this with send_audio_frame, events()/results(), and stop). | |
| recognize_file | Non-streaming call based on a local file: automatically sends the audio in chunks and drives the task to completion, yielding server events one by one. The file to be recognized must have read permissions. | |
| start | Starts speech recognition: establishes the connection, submits the task, and waits for task-started. It is automatically called when entering the with context. Audio can be sent only after a successful start. | |
| send_audio_frame | Pushes an audio stream. The audio stream pushed each time should not be too large or too small. We recommend that each audio packet has a duration of about 100 ms and a size between 1 KB and 16 KB.You can obtain the recognition results through event iteration (events()/results()). | |
| stop | Stops speech recognition: it only notifies the server that the input is complete. The receiver thread must continue to consume events() until the task-finished event. | |
| events / results | Iterates server events: events() yields raw event dicts (the recognized text is located in payload.output.sentence); results() yields only recognition result frames, returning typed RecognitionResult objects with heartbeat frames filtered out. | |
| close | Closes the recognition session. Automatically called when exiting the with context. | |
| request_id / first_package_delay / last_package_delay | Gets the request_id, the first packet delay (the latency from sending the first audio packet to receiving the first recognition result), and the last packet delay (the time taken from sending the stop instruction to receiving the last recognition result). Use this after the task is completed. The values remain readable after the session is closed. |
Event iteration (events()/results())
During a bidirectional streaming call, the server returns key process information and data to the client through events. You need to iterate the session's events() (raw event dicts) or results() (typed recognition result frames) in a receiver thread to process the information and data returned by the server.
Click to view the example
Click to view the example
| Event/iteration method | Output | Return value | Description |
|---|---|---|---|
task-started event | None | Read internally by start(). It is stored in last_response when the start call returns and is not yielded again by events(). | |
event: a server event dict | Iterator | Yields an event whenever the server sends a response. The recognized text is located in payload.output.sentence. Heartbeat or empty-text events may also be yielded and can be ignored as needed. Consume the iterator continuously from one receiver thread, and do not create multiple concurrent readers. | |
result: RecognitionResult | Iterator | Yields only recognition result frames (heartbeat frames filtered out). The iteration ends after all recognition results are returned. | |
DashScopeError exception | None | Raised in the sending/receiving thread when an exception occurs. You can catch it with try/except. | |
task-finished event | None | Yielded by events() after the server has completed all recognition, and the iteration then ends. You must still read this event after calling stop(). |
Response results
Recognition result (RecognitionResult)
RecognitionResult represents a recognition result frame yielded by iterating the session's results(). It can be used in both bidirectional streaming calls and non-streaming calls. events() yields raw event dicts; use results() if you need this typed view.
| Member method | Method signature | Description |
|---|---|---|
| get_sentence | Gets the currently recognized sentence and timestamp information. This method returns a Dict[str, Any] type (single-sentence information).For more information, see Sentence. | |
| request_id | The request_id of the current event. | |
| is_sentence_end | Determines whether the current sentence has ended (True when the sentence carries end_time). |
Sentence (Sentence)
The members of the Sentence class are as follows:
Parameter | Type | Description |
|---|---|---|
begin_time | int | The start time of the sentence, in ms. |
end_time | int | The end time of the sentence, in ms. |
text | str | The recognized text. |
words | A list of Word timestamp information (Word) | Word timestamp information. |
emo_tag | str | The emotion of the current sentence:
Emotion recognition has the following constraints:
|
emo_confidence | float | The confidence level of the recognized emotion for the current sentence. The value ranges from 0.0 to 1.0. A larger value indicates a higher confidence level. Emotion recognition has the following constraints:
|
Word timestamp information (Word)
The members of the Word class are as follows:
Parameter | Type | Description |
|---|---|---|
begin_time | int | The start time of the word, in ms. |
end_time | int | The end time of the word, in ms. |
text | str | The word. |
punctuation | str | The punctuation. |