This topic describes the Python APIs, callbacks, and data types of AOQ Client SDK for Linux.
aoq_client_sdk Python module. All callbacks are triggered on native threads, and you must ensure thread safety yourself.
API index
Engine lifecycle
API | Description |
|---|---|
create_engine | Create the engine instance (singleton) |
destroy | Destroy the engine instance |
get_version | Get the SDK version |
connect | Connect to the relay server |
disconnect | Disconnect from the server |
Audio device management
API | Description |
|---|---|
start_audio_capture | Start audio capture (empty implementation on Linux; does not open a microphone) |
stop_audio_capture | Stop audio capture (no actual effect on Linux) |
mute_audio_capture | Mute or unmute audio capture (device capture is not available on Linux) |
start_audio_player | Start audio playback (empty implementation on Linux; does not open a speaker) |
stop_audio_player | Stop audio playback (no actual effect on Linux) |
pause_audio_player | Pause audio playback (device playback is not available on Linux) |
resume_audio_player | Resume audio playback (device playback is not available on Linux) |
interrupt_audio_player | Interrupt the current audio session |
Audio encoding configuration
API | Description |
|---|---|
set_audio_encoder_config | Set audio encoding parameters |
set_audio_decoder_config | Set audio decoding parameters |
Video device management
API | Description |
|---|---|
start_video_capture | Start video capture (only external capture mode is supported on Linux) |
stop_video_capture | Stop video capture (applies only to external capture mode on Linux) |
set_local_view | Set or remove the local video rendering window (no rendering implementation on Linux) |
set_remote_view | Set or remove the remote video rendering window (no rendering implementation on Linux) |
Video codec and external input
API | Description |
|---|---|
set_video_encoder_config | Set video encoding parameters |
set_video_decoder_config | Set video decoding parameters |
push_external_video_frame | Push an externally captured video frame |
push_external_video_encoded_frame | Push an externally encoded video frame |
Media stream control
API | Description |
|---|---|
enable_send_media_stream | Enable or disable sending of a local media stream |
Audio file playback
API | Description |
|---|---|
start_audio_file | Start playing a local audio file to the publishing stream |
stop_audio_file | Stop audio file playback |
pause_audio_file | Pause audio file playback |
resume_audio_file | Resume audio file playback |
get_audio_file_duration | Query the total duration of the audio file |
get_audio_file_current_position | Query the current playback position of the audio file |
set_audio_file_position_millis | Set the playback position of the audio file (seek) |
set_audio_file_volume | Set the volume of the audio file |
get_audio_file_volume | Query the current volume of the audio file |
External audio streams
API | Description |
|---|---|
add_audio_external_stream | Add an external audio stream |
remove_audio_external_stream | Remove an external audio stream |
push_audio_external_stream_data | Feed external audio PCM data |
set_audio_external_stream_volume | Set the volume of an external audio stream |
get_audio_external_stream_volume | Query the volume of an external audio stream |
clear_audio_external_stream_buffer | Clear the buffer of an external audio stream |
Real-time messaging
API | Description |
|---|---|
send_data_msg | Send a real-time data message |
Audio frame callbacks
API | Description |
|---|---|
set_audio_frame_observer | Set the audio frame data callback listener |
enable_audio_frame_observer | Enable or disable the audio frame callback at a specified position |
Video frame callbacks
API | Description |
|---|---|
set_video_frame_observer | Set the video frame data callback listener |
enable_video_frame_observer | Enable or disable the video frame callback at a specified position |
Callback interfaces
Callback | Description |
|---|---|
on_error | Engine error callback |
on_connection_status_change | Connection status change callback |
on_data_msg | Callback for a received real-time data message |
IAudioFrameObserver | Base class for audio frame data listeners |
IVideoFrameObserver | Base class for video frame data listeners |
Utility functions
API | Description |
|---|---|
load_library | Manually specify and load the native shared library |
API details
Engine lifecycle
create_engine
Creates the engine instance (class method). The SDK holds the engine as a global singleton, so calling this method again returns the existing instance. After destroy, you must call create_engine again before you can continue to use the engine.
Parameters | Type | Description |
|---|---|---|
config | AoqCreateConfig | Engine creation configuration |
listener | AoqEngineEventListener | Engine event callback listener |
destroy
Destroys the engine instance (class method) and releases all resources.
get_version
Gets the current SDK version (static method).
connect
Connects to the relay server. The application server obtains temporary AOQ connection parameters based on the protocol in use and sends them to the client. For more information, see Token authentication.
Parameters | Type | Description |
|---|---|---|
config | AoqConnectConfig | Connection configuration, which includes the token, SID, and the list of relay access points |
disconnect
Disconnects from the server and releases the resources associated with the connection.
Audio device management
start_audio_capture
stop_audio_capture
mute_audio_capture
start_audio_player
stop_audio_player / pause_audio_player / resume_audio_player
interrupt_audio_player
Audio encoding configuration
Video device management
start_video_capture, set is_external to True and provide frames by calling push_external_video_frame. set_local_view, set_remote_view, and AoqVideoCanvas.view have no actual effect on Linux. To preview video, obtain frames through a video frame callback and render them yourself.
Video codec and external input
- For set_video_decoder_config, only the track_type/codec_type/width/height/fps/bitrate fields take effect.
- push_external_video_frame is consumed only after start_video_capture(is_external=True). For packed formats (NV12/NV21/BGRA/RGBA), fill
frame.data; for the three I420 planes, fillframe.data_y/u/vand the corresponding stride. If the buffer is full, it returns AoqErrorCode.VIDEO_EXTERNAL_BUFFER_FULL(210). - push_external_video_encoded_frame requires start_video_capture(is_external=True) and set_video_encoder_config(codec_type=VIDEO_JPEG), and takes the bypass path without re-encoding.
Media stream control
Audio file playback
External audio streams
Real-time messaging
Audio frame callbacks
Video frame callbacks
Callback interfaces
AoqEngineEventListener
Base class for engine event callbacks. All methods are optional overrides with default empty implementations. Callbacks are triggered on native threads, and you must ensure thread safety yourself.
on_error
on_connection_status_change
on_data_msg
IAudioFrameObserver
Base class for audio frame data listeners. All methods are optional overrides with default empty implementations.
AoqAudioFrameData.data is already a bytes copy and can be safely used asynchronously.
IVideoFrameObserver
Base class for video frame data listeners. All methods are optional overrides and return False by default.
Utility functions
load_library
libAoqClientSdk.so is searched for in the following order: the AOQ_CLIENT_SDK_LIB environment variable, the directory of the module, the lib directory at the same level, and the default system paths. If the loading fails, an OSError is thrown. A typical cause is that a dependent library is not in LD_LIBRARY_PATH.
Data types and enumerations
All data types are Python dataclasses. You can construct them directly and assign values to the fields.
General types
AoqCreateConfig
Field | Type | Default value | Description |
|---|---|---|---|
work_dir | str | "" | SDK working directory |
enable_dump_audio | bool | False | Specifies whether to enable audio dump (for debugging) |
extras | str | "" | Extended parameter string |
AoqConnectConfig
Field | Type | Default value | Description |
|---|---|---|---|
token | str | "" | Connection authentication token |
sid | str | "" | Session ID |
certificate | str | "" | Server certificate fingerprint |
relay_endpoints | List[AoqRelayEndpoint] | None | List of relay access points |
workspace_id_hash | str | "" | Workspace ID hash |
publish_tracks | List[AoqTrackParam] | None | List of local published tracks |
subscribe_tracks | List[AoqTrackParam] | None | List of local subscribed tracks |
AoqRelayEndpoint
Field | Type | Default value | Description |
|---|---|---|---|
endpoint | str | "" | Domain name or IP address of the relay server |
port | int | 0 | Port of the relay server |
route_index | int | -1 | Path index, which is aligned with routeIndex in SDKs for other platforms. If it is less than 0, the SDK automatically fills it in based on the array subscript. |
AoqTrackParam
Field | Type | Default value | Description |
|---|---|---|---|
track_type | AoqTrackType | AoqTrackType.AUDIO | Track type |
AoqDataMsg
Field | Type | Default value | Description |
|---|---|---|---|
data | bytes | b"" | Message data (a byte string) |
Enumerations
AoqErrorCode
Enum value | Value | Description |
|---|---|---|
OK | 0 | Success |
PARAM_INVALID | 1 | Invalid parameter |
STATE_INVALID | 2 | Invalid state |
AUDIO | 100 | Generic audio error |
AUDIO_EXTERNAL_BUFFER_FULL | 110 | External audio buffer is full |
AUDIO_DEVICE | 120 | Generic audio device error |
AUDIO_DEVICE_RECORDING_AUTH_FAILED | 121 | Recording permission not granted |
AUDIO_DEVICE_RECORDING_OCCUPIED | 122 | Recording device is in use |
AUDIO_DEVICE_RECORDING_START_FAIL | 124 | Failed to start recording |
AUDIO_DEVICE_PLAYOUT_OCCUPIED | 125 | Playback device is in use |
AUDIO_DEVICE_PLAYOUT_START_FAIL | 127 | Failed to start playback |
VIDEO | 200 | Generic video error |
VIDEO_EXTERNAL_BUFFER_FULL | 210 | External video buffer is full |
AoqConnectionStatus
Enum value | Value | Description |
|---|---|---|
DISCONNECTED | 0 | Disconnected |
CONNECTING | 1 | Connecting |
CONNECTED | 2 | Connected |
FAILED | 3 | Connection failed |
AoqTrackType
Enum value | Value | Description |
|---|---|---|
AUDIO | 0 | Audio track |
VIDEO | 1 | Video track |
DATA | 2 | Data message track |
AoqEncoderType
Enum value | Value | Description |
|---|---|---|
UNKNOWN | 0 | Unknown format |
AUDIO_PCM | 1 | Audio PCM |
AUDIO_OPUS | 2 | Audio Opus |
VIDEO_H264 | 3 | Video H.264 |
VIDEO_JPEG | 4 | Video JPEG |
DATA_TEXT | 5 | Message text |
AoqMirrorMode
Enum value | Value | Description |
|---|---|---|
DISABLED | 0 | Disable mirroring |
ENABLED | 1 | Enable mirroring |
AoqOrientationMode
Enum value | Value | Description |
|---|---|---|
AUTO | 0 | Auto-fit |
PORTRAIT | 1 | Portrait |
LANDSCAPE | 2 | Landscape |
AoqRenderMode
Enum value | Value | Description |
|---|---|---|
AUTO | 0 | Adaptive mode |
STRETCH | 1 | Stretch mode |
FILL | 2 | Fill mode |
CROP | 3 | Crop mode |
AoqVideoPixelFormat
Enum value | Value | Description |
|---|---|---|
UNKNOWN | 0 | Unknown format |
I420 | 1 | I420 (YUV planar format) |
NV12 | 2 | NV12 (YUV semi-planar format) |
NV21 | 3 | NV21 (YUV semi-planar format) |
BGRA | 4 | BGRA (32-bit) |
RGBA | 5 | RGBA (32-bit) |
AoqCameraDirection
Enum value | Value | Description |
|---|---|---|
FRONT | 0 | Front camera (platform-generic enum value; camera capture is not supported on Linux) |
BACK | 1 | Rear camera (platform-generic enum value; camera capture is not supported on Linux) |
Audio types
AoqAudioCaptureConfig
Field | Type | Default value | Description |
|---|---|---|---|
is_external | bool | False | Specifies whether to use external capture mode |
channel | int | 1 | Number of channels (mono by default) |
AoqAudioPlaybackConfig
Field | Type | Default value | Description |
|---|---|---|---|
is_external | bool | False | Specifies whether to use external playback mode |
channel | int | 1 | Number of channels (mono by default) |
AoqAudioCodecConfig
Field | Type | Default value | Description |
|---|---|---|---|
track_type | AoqTrackType | AUDIO | Track type |
codec_type | AoqEncoderType | AUDIO_PCM | Codec format |
sample_rate | int | 48000 | Sample rate, in Hz |
channel | int | 1 | Number of channels |
bitrate | int | 32000 | Bitrate, in bit/s |
AoqAudioFileMixConfig
Field | Type | Default value | Description |
|---|---|---|---|
file_name | str | "" | File name (including the path). It cannot be empty. |
cycles | int | -1 | Number of loops. -1 indicates unlimited looping. |
start_pos_ms | int | 0 | Start playback position, in milliseconds |
publish_volume | int | 100 | Publishing volume. Valid values: 0 to 100. |
playout_volume | int | 100 | Playback volume. Valid values: 0 to 100. |
AoqAudioExternalStreamConfig
Field | Type | Default value | Description |
|---|---|---|---|
track_type | AoqTrackType | AUDIO | Audio track type |
codec_type | AoqEncoderType | AUDIO_PCM | Audio stream format |
channels | int | 1 | Number of channels |
sample_rate | int | 48000 | Sample rate, in Hz |
playout_volume | int | 100 | Playback volume. Valid values: 0 to 100. |
publish_volume | int | 100 | Publishing volume. Valid values: 0 to 100. |
max_buffer_duration | int | 1000 | Maximum buffer duration, in milliseconds |
enable_3a | bool | False | Specifies whether to apply 3A processing to the input PCM |
AoqAudioFrameData
Raw audio data, which is used for external input or observer callbacks.
Field | Type | Default value | Description |
|---|---|---|---|
data | bytes | b"" | Raw audio PCM data (a bytes copy in callbacks) |
num_of_samples | int | 0 | Number of samples (per channel) |
bytes_per_sample | int | 0 | Bytes per sample |
num_of_channels | int | 0 | Number of channels |
samples_per_sec | int | 0 | Number of samples per second (sample rate) |
push_sequence | int | 0 | PCM input round |
time_stamp | int | 0 | Timestamp |
auto_gen_mute | bool | False | True indicates silent data generated by the SDK |
AoqAudioObserverConfig
Field | Type | Default value | Description |
|---|---|---|---|
sample_rate | int | 48000 | Sample rate of the callback audio, in Hz |
channels | int | 1 | Number of audio channels in the callback |
mode | AoqAudioObserverMode | READ_ONLY | Read-write mode |
AoqAudioStreamDirection
Enum value | Value | Description |
|---|---|---|
PUBLISH | 0 | Publishing stream |
PLAYOUT | 1 | Playback stream (subscribing stream) |
AoqAudioExternalStreamToggle
Enum value | Value | Description |
|---|---|---|
NORMAL | 0 | Normal state |
PAUSE | 1 | Paused state |
AoqAudioSource
Enum value | Value | Description |
|---|---|---|
CAPTURED | 0 | Captured audio data |
PROCESS_CAPTURED | 1 | Audio data after 3A processing |
PUBLISH | 2 | Audio data to be published |
PLAYBACK | 3 | Audio data to be played |
AoqAudioObserverMode
Enum value | Value | Description |
|---|---|---|
READ_ONLY | 0 | Read-only mode |
READ_WRITE | 1 | Read-write mode |
Video types
AoqVideoCaptureConfig
Field | Type | Default value | Description |
|---|---|---|---|
width | int | 1280 | Capture width, in pixels. This parameter is ineffective when is_external=True. |
height | int | 720 | Capture height, in pixels. This parameter is ineffective when is_external=True. |
fps | int | 15 | Capture frame rate. This parameter is ineffective when is_external=True. |
is_external | bool | False | Specifies whether to use external capture. Linux supports only True. After you call start_video_capture, provide frames by calling push_external_video_frame. |
camera_direction | AoqCameraDirection | FRONT | Camera direction for mobile platforms. Linux has no corresponding capability. Keep the value FRONT. |
AoqVideoCodecConfig
Video codec parameters, which are shared by encoding and decoding. For decoding, only track_type/codec_type/width/height/fps/bitrate take effect, and the other fields are used only for encoding.
Field | Type | Default value | Description |
|---|---|---|---|
track_type | AoqTrackType | VIDEO | Track type |
codec_type | AoqEncoderType | VIDEO_H264 | Codec format |
width | int | 540 | Encoding width, in pixels |
height | int | 960 | Encoding height, in pixels |
fps | int | 5 | Encoding frame rate |
bitrate | int | 500000 | Target bitrate, in bit/s |
min_bitrate | int | 128000 | Minimum bitrate, in bit/s |
keyframe_interval | int | 2 | Keyframe interval, in seconds |
mirror_mode | AoqMirrorMode | DISABLED | Mirror mode |
orientation_mode | AoqOrientationMode | AUTO | Video orientation mode |
AoqVideoCanvas
Field | Type | Default value | Description |
|---|---|---|---|
view | int | 0 | Rendering window handle. Linux has no rendering backend, so this field has no actual effect. Keep the value 0. |
render_mode | AoqRenderMode | AUTO | Rendering mode. Linux has no rendering backend, so this field has no actual effect. |
AoqVideoFrame
External video frame or video frame callback data. When you use a packed format (NV12/NV21/BGRA/RGBA), fill in data. When you use the three I420 planes, fill in data_y/u/v and the corresponding stride. The two methods are mutually exclusive.
Field | Type | Default value | Description |
|---|---|---|---|
format | AoqVideoPixelFormat | UNKNOWN | Pixel format |
width | int | 0 | Video width, in pixels |
height | int | 0 | Video height, in pixels |
data | bytes | b"" | Packed-format data (NV12/NV21/BGRA/RGBA) |
data_y | bytes | b"" | I420 Y plane data |
data_u | bytes | b"" | I420 U plane data |
data_v | bytes | b"" | I420 V plane data |
stride_y | int | 0 | Stride of the Y plane |
stride_u | int | 0 | Stride of the U plane |
stride_v | int | 0 | Stride of the V plane |
time_stamp | int | 0 | Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock. |
AoqVideoEncodedFrame
An externally encoded video frame, such as a JPEG frame.
Field | Type | Default value | Description |
|---|---|---|---|
codec | AoqEncoderType | VIDEO_JPEG | Codec format |
data | bytes | b"" | Encoded data |
width | int | 0 | Width, in pixels |
height | int | 0 | Height, in pixels |
time_stamp | int | 0 | Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock. |
AoqVideoObserverConfig
Field | Type | Default value | Description |
|---|---|---|---|
format | AoqVideoPixelFormat | I420 | Expected pixel format of the callback data |
alignment | AoqVideoObserverAlignment | DEFAULT | Width alignment policy |
mode | AoqVideoObserverMode | READ_ONLY | Read-write mode |
mirror_applied | bool | False | Specifies whether to mirror the callback data |
AoqVideoSource
Enum value | Value | Description |
|---|---|---|
CAPTURED | 0 | Captured video data before preprocessing |
PRE_ENCODE | 1 | Video data before encoding, after preprocessing |
REMOTE | 2 | Remote video data after decoding and before rendering |
AoqVideoObserverMode
Enum value | Value | Description |
|---|---|---|
READ_ONLY | 0 | Read-only mode |
READ_WRITE | 1 | Read-write mode |
AoqVideoObserverAlignment
Enum value | Value | Description |
|---|---|---|
DEFAULT | 0 | Default alignment |
EVEN | 1 | Even-number alignment |
ALIGN_4 | 2 | 4-byte alignment |
ALIGN_8 | 3 | 8-byte alignment |
ALIGN_16 | 4 | 16-byte alignment |