Skip to main content
AOQ SDK overview

AOQ Client SDK Linux Python API reference

This topic describes the Python APIs, callbacks, and data types of AOQ Client SDK for Linux.

The Linux platform provides APIs through the aoq_client_sdk Python module. All callbacks are triggered on native threads, and you must ensure thread safety yourself.

API index

Engine lifecycle

API

Description

create_engine

Create the engine instance (singleton)

destroy

Destroy the engine instance

get_version

Get the SDK version

connect

Connect to the relay server

disconnect

Disconnect from the server

Audio device management

API

Description

start_audio_capture

Start audio capture (empty implementation on Linux; does not open a microphone)

stop_audio_capture

Stop audio capture (no actual effect on Linux)

mute_audio_capture

Mute or unmute audio capture (device capture is not available on Linux)

start_audio_player

Start audio playback (empty implementation on Linux; does not open a speaker)

stop_audio_player

Stop audio playback (no actual effect on Linux)

pause_audio_player

Pause audio playback (device playback is not available on Linux)

resume_audio_player

Resume audio playback (device playback is not available on Linux)

interrupt_audio_player

Interrupt the current audio session

Audio encoding configuration

API

Description

set_audio_encoder_config

Set audio encoding parameters

set_audio_decoder_config

Set audio decoding parameters

Video device management

API

Description

start_video_capture

Start video capture (only external capture mode is supported on Linux)

stop_video_capture

Stop video capture (applies only to external capture mode on Linux)

set_local_view

Set or remove the local video rendering window (no rendering implementation on Linux)

set_remote_view

Set or remove the remote video rendering window (no rendering implementation on Linux)

Video codec and external input

API

Description

set_video_encoder_config

Set video encoding parameters

set_video_decoder_config

Set video decoding parameters

push_external_video_frame

Push an externally captured video frame

push_external_video_encoded_frame

Push an externally encoded video frame

Media stream control

API

Description

enable_send_media_stream

Enable or disable sending of a local media stream

Audio file playback

API

Description

start_audio_file

Start playing a local audio file to the publishing stream

stop_audio_file

Stop audio file playback

pause_audio_file

Pause audio file playback

resume_audio_file

Resume audio file playback

get_audio_file_duration

Query the total duration of the audio file

get_audio_file_current_position

Query the current playback position of the audio file

set_audio_file_position_millis

Set the playback position of the audio file (seek)

set_audio_file_volume

Set the volume of the audio file

get_audio_file_volume

Query the current volume of the audio file

External audio streams

API

Description

add_audio_external_stream

Add an external audio stream

remove_audio_external_stream

Remove an external audio stream

push_audio_external_stream_data

Feed external audio PCM data

set_audio_external_stream_volume

Set the volume of an external audio stream

get_audio_external_stream_volume

Query the volume of an external audio stream

clear_audio_external_stream_buffer

Clear the buffer of an external audio stream

Real-time messaging

API

Description

send_data_msg

Send a real-time data message

Audio frame callbacks

API

Description

set_audio_frame_observer

Set the audio frame data callback listener

enable_audio_frame_observer

Enable or disable the audio frame callback at a specified position

Video frame callbacks

API

Description

set_video_frame_observer

Set the video frame data callback listener

enable_video_frame_observer

Enable or disable the video frame callback at a specified position

Callback interfaces

Callback

Description

on_error

Engine error callback

on_connection_status_change

Connection status change callback

on_data_msg

Callback for a received real-time data message

IAudioFrameObserver

Base class for audio frame data listeners

IVideoFrameObserver

Base class for video frame data listeners

Utility functions

API

Description

load_library

Manually specify and load the native shared library

API details

Engine lifecycle

create_engine

Creates the engine instance (class method). The SDK holds the engine as a global singleton, so calling this method again returns the existing instance. After destroy, you must call create_engine again before you can continue to use the engine.
@classmethod
def create_engine(cls, config: AoqCreateConfig, listener: AoqEngineEventListener) -> "AoqClientEngine"

Parameters

Type

Description

config

AoqCreateConfig

Engine creation configuration

listener

AoqEngineEventListener

Engine event callback listener

Returns: The AoqClientEngine engine instance. If the creation fails, a RuntimeError is thrown.

destroy

Destroys the engine instance (class method) and releases all resources.
@classmethod
def destroy(cls) -> int
Returns: 0 on success; non-zero on failure.

get_version

Gets the current SDK version (static method).
@staticmethod
def get_version() -> str
Returns: The version string, such as "1.0.0".

connect

Connects to the relay server. The application server obtains temporary AOQ connection parameters based on the protocol in use and sends them to the client. For more information, see Token authentication.
def connect(self, config: AoqConnectConfig) -> int

Parameters

Type

Description

config

AoqConnectConfig

Connection configuration, which includes the token, SID, and the list of relay access points

Returns: 0 indicates that the call is dispatched and runs asynchronously; non-zero indicates that parameter validation failed.

disconnect

Disconnects from the server and releases the resources associated with the connection.
def disconnect(self) -> int
Returns: 0 indicates that the call is dispatched and runs asynchronously; non-zero indicates a failure.

Audio device management

start_audio_capture

def start_audio_capture(self, config: AoqAudioCaptureConfig) -> int
On Linux, this method is an empty implementation. It returns 0 but does not open a sound card or microphone. Provide audio input through an external audio stream.

stop_audio_capture

def stop_audio_capture(self) -> int
This method has no actual effect because the Linux build does not set the audio capture device state.

mute_audio_capture

def mute_audio_capture(self, mute: bool) -> int
The Linux build does not support microphone capture. This method cannot control an actual audio capture device.

start_audio_player

def start_audio_player(self, config: AoqAudioPlaybackConfig) -> int
On Linux, this method is an empty implementation. It returns 0 but does not open a sound card or speaker. Obtain audio output through an audio frame callback and play it yourself.

stop_audio_player / pause_audio_player / resume_audio_player

def stop_audio_player(self) -> int
def pause_audio_player(self, fade_ms: int = 0) -> int
def resume_audio_player(self, fade_ms: int = 0) -> int
The Linux build does not support speaker playback, so these methods cannot control an actual audio playback device. fade_ms specifies the fade-out or fade-in duration in milliseconds. 0 indicates immediate execution.

interrupt_audio_player

def interrupt_audio_player(self, track_type: int, fade_ms: int = 0) -> int
Interrupts the current audio session.

Audio encoding configuration

def set_audio_encoder_config(self, config: AoqAudioCodecConfig) -> int
def set_audio_decoder_config(self, config: AoqAudioCodecConfig) -> int

Video device management

def start_video_capture(self, config: AoqVideoCaptureConfig) -> int
def stop_video_capture(self) -> int
def set_local_view(self, track_type: int, canvas: Optional[AoqVideoCanvas]) -> int
def set_remote_view(self, track_type: int, canvas: Optional[AoqVideoCanvas]) -> int
The Linux build does not support camera capture and has no video rendering backend. When you call start_video_capture, set is_external to True and provide frames by calling push_external_video_frame. set_local_view, set_remote_view, and AoqVideoCanvas.view have no actual effect on Linux. To preview video, obtain frames through a video frame callback and render them yourself.

Video codec and external input

def set_video_encoder_config(self, config: AoqVideoCodecConfig) -> int
def set_video_decoder_config(self, config: AoqVideoCodecConfig) -> int
def push_external_video_frame(self, frame: AoqVideoFrame, track_type: AoqTrackType = AoqTrackType.VIDEO) -> int
def push_external_video_encoded_frame(self, track_type: AoqTrackType, frame: AoqVideoEncodedFrame) -> int
Notes:
  • For set_video_decoder_config, only the track_type/codec_type/width/height/fps/bitrate fields take effect.
  • push_external_video_frame is consumed only after start_video_capture(is_external=True). For packed formats (NV12/NV21/BGRA/RGBA), fill frame.data; for the three I420 planes, fill frame.data_y/u/v and the corresponding stride. If the buffer is full, it returns AoqErrorCode.VIDEO_EXTERNAL_BUFFER_FULL(210).
  • push_external_video_encoded_frame requires start_video_capture(is_external=True) and set_video_encoder_config(codec_type=VIDEO_JPEG), and takes the bypass path without re-encoding.

Media stream control

def enable_send_media_stream(self, track_type: AoqTrackType, enable: bool) -> int
After initialization, disable sending separately for each required track by calling enable_send_media_stream(AoqTrackType.AUDIO, False) and enable_send_media_stream(AoqTrackType.VIDEO, False) as needed. After on_connection_status_change(CONNECTED) is received, enable each track in a separate call.

Audio file playback

def start_audio_file(self, file_id: str, config: AoqAudioFileMixConfig) -> int
def stop_audio_file(self, file_id: str) -> int
def pause_audio_file(self, file_id: str) -> int
def resume_audio_file(self, file_id: str) -> int
def get_audio_file_duration(self, file_id: str) -> int
def get_audio_file_current_position(self, file_id: str) -> int
def set_audio_file_position_millis(self, file_id: str, position_ms: int) -> int
def set_audio_file_volume(self, file_id: str, type_: AoqAudioStreamDirection, volume: int) -> int
def get_audio_file_volume(self, file_id: str, type_: AoqAudioStreamDirection) -> int

External audio streams

def add_audio_external_stream(self, stream_id: str, config: AoqAudioExternalStreamConfig) -> int
def remove_audio_external_stream(self, stream_id: str) -> int
def push_audio_external_stream_data(self, stream_id: str, data: AoqAudioFrameData) -> int
def set_audio_external_stream_volume(self, stream_id: str, type_: AoqAudioStreamDirection, volume: int) -> int
def get_audio_external_stream_volume(self, stream_id: str, type_: AoqAudioStreamDirection) -> int
def clear_audio_external_stream_buffer(self, stream_id: str, fadeout_ms: int = -1) -> None
Note: When the buffer is full, push_audio_external_stream_data returns AoqErrorCode.AUDIO_EXTERNAL_BUFFER_FULL(110). We recommend that you wait about 20 ms and then retry the same frame.

Real-time messaging

def send_data_msg(self, msg: AoqDataMsg) -> int

Audio frame callbacks

def set_audio_frame_observer(self, observer: Optional[IAudioFrameObserver]) -> int
def enable_audio_frame_observer(self, enabled: bool, audio_source: AoqAudioSource, config: AoqAudioObserverConfig) -> int
Pass None to set_audio_frame_observer to unregister the observer.

Video frame callbacks

def set_video_frame_observer(self, observer: Optional[IVideoFrameObserver]) -> int
def enable_video_frame_observer(self, enabled: bool, video_source: AoqVideoSource, config: AoqVideoObserverConfig) -> int
Pass None to set_video_frame_observer to unregister the observer.

Callback interfaces

AoqEngineEventListener

Base class for engine event callbacks. All methods are optional overrides with default empty implementations. Callbacks are triggered on native threads, and you must ensure thread safety yourself.
class AoqEngineEventListener:
    def on_error(self, code: int, message: str) -> None: ...
    def on_connection_status_change(self, status: AoqConnectionStatus) -> None: ...
    def on_data_msg(self, msg: AoqDataMsg) -> None: ...

on_error

def on_error(self, code: int, message: str) -> None
Engine error callback. code corresponds to an AoqErrorCode enum value.

on_connection_status_change

def on_connection_status_change(self, status: AoqConnectionStatus) -> None
Connection status change callback. State transitions: DISCONNECTED -> CONNECTING -> CONNECTED/FAILED -> DISCONNECTED.

on_data_msg

def on_data_msg(self, msg: AoqDataMsg) -> None
Callback for a received real-time data message.

IAudioFrameObserver

Base class for audio frame data listeners. All methods are optional overrides with default empty implementations.
class IAudioFrameObserver:
    def on_captured_audio_frame(self, data: AoqAudioFrameData) -> None: ...
    def on_process_captured_audio_frame(self, data: AoqAudioFrameData) -> None: ...
    def on_publish_audio_frame(self, track_type: AoqTrackType, data: AoqAudioFrameData) -> None: ...
    def on_playback_audio_frame(self, data: AoqAudioFrameData) -> None: ...
The callback is triggered on the native audio thread. Do not perform any time-consuming operations in it. AoqAudioFrameData.data is already a bytes copy and can be safely used asynchronously.

IVideoFrameObserver

Base class for video frame data listeners. All methods are optional overrides and return False by default.
class IVideoFrameObserver:
    def on_captured_video_frame(self, frame: AoqVideoFrame) -> bool: ...
    def on_pre_encode_video_frame(self, track_type: AoqTrackType, frame: AoqVideoFrame) -> bool: ...
    def on_remote_video_frame(self, track_type: AoqTrackType, frame: AoqVideoFrame) -> bool: ...
The callback is triggered on the native video thread. Do not perform any time-consuming operations in it. The pixel data in the frame is already a bytes copy. The Python layer currently uses copy semantics and does not support writing modifications back, so we recommend that you always return False.

Utility functions

load_library

def load_library(path: Optional[str] = None) -> ctypes.CDLL
Loads the native shared library. You usually do not need to call this method manually, because the library is loaded automatically when you first use the engine. If path is empty, libAoqClientSdk.so is searched for in the following order: the AOQ_CLIENT_SDK_LIB environment variable, the directory of the module, the lib directory at the same level, and the default system paths. If the loading fails, an OSError is thrown. A typical cause is that a dependent library is not in LD_LIBRARY_PATH.

Data types and enumerations

All data types are Python dataclasses. You can construct them directly and assign values to the fields.

General types

AoqCreateConfig

Field

Type

Default value

Description

work_dir

str

""

SDK working directory

enable_dump_audio

bool

False

Specifies whether to enable audio dump (for debugging)

extras

str

""

Extended parameter string

AoqConnectConfig

Field

Type

Default value

Description

token

str

""

Connection authentication token

sid

str

""

Session ID

certificate

str

""

Server certificate fingerprint

relay_endpoints

List[AoqRelayEndpoint]

None

List of relay access points

workspace_id_hash

str

""

Workspace ID hash

publish_tracks

List[AoqTrackParam]

None

List of local published tracks

subscribe_tracks

List[AoqTrackParam]

None

List of local subscribed tracks

AoqRelayEndpoint

Field

Type

Default value

Description

endpoint

str

""

Domain name or IP address of the relay server

port

int

0

Port of the relay server

route_index

int

-1

Path index, which is aligned with routeIndex in SDKs for other platforms. If it is less than 0, the SDK automatically fills it in based on the array subscript.

AoqTrackParam

Field

Type

Default value

Description

track_type

AoqTrackType

AoqTrackType.AUDIO

Track type

AoqDataMsg

Field

Type

Default value

Description

data

bytes

b""

Message data (a byte string)

Enumerations

AoqErrorCode

Enum value

Value

Description

OK

0

Success

PARAM_INVALID

1

Invalid parameter

STATE_INVALID

2

Invalid state

AUDIO

100

Generic audio error

AUDIO_EXTERNAL_BUFFER_FULL

110

External audio buffer is full

AUDIO_DEVICE

120

Generic audio device error

AUDIO_DEVICE_RECORDING_AUTH_FAILED

121

Recording permission not granted

AUDIO_DEVICE_RECORDING_OCCUPIED

122

Recording device is in use

AUDIO_DEVICE_RECORDING_START_FAIL

124

Failed to start recording

AUDIO_DEVICE_PLAYOUT_OCCUPIED

125

Playback device is in use

AUDIO_DEVICE_PLAYOUT_START_FAIL

127

Failed to start playback

VIDEO

200

Generic video error

VIDEO_EXTERNAL_BUFFER_FULL

210

External video buffer is full

AoqConnectionStatus

Enum value

Value

Description

DISCONNECTED

0

Disconnected

CONNECTING

1

Connecting

CONNECTED

2

Connected

FAILED

3

Connection failed

AoqTrackType

Enum value

Value

Description

AUDIO

0

Audio track

VIDEO

1

Video track

DATA

2

Data message track

AoqEncoderType

Enum value

Value

Description

UNKNOWN

0

Unknown format

AUDIO_PCM

1

Audio PCM

AUDIO_OPUS

2

Audio Opus

VIDEO_H264

3

Video H.264

VIDEO_JPEG

4

Video JPEG

DATA_TEXT

5

Message text

AoqMirrorMode

Enum value

Value

Description

DISABLED

0

Disable mirroring

ENABLED

1

Enable mirroring

AoqOrientationMode

Enum value

Value

Description

AUTO

0

Auto-fit

PORTRAIT

1

Portrait

LANDSCAPE

2

Landscape

AoqRenderMode

Enum value

Value

Description

AUTO

0

Adaptive mode

STRETCH

1

Stretch mode

FILL

2

Fill mode

CROP

3

Crop mode

AoqVideoPixelFormat

Enum value

Value

Description

UNKNOWN

0

Unknown format

I420

1

I420 (YUV planar format)

NV12

2

NV12 (YUV semi-planar format)

NV21

3

NV21 (YUV semi-planar format)

BGRA

4

BGRA (32-bit)

RGBA

5

RGBA (32-bit)

AoqCameraDirection

Enum value

Value

Description

FRONT

0

Front camera (platform-generic enum value; camera capture is not supported on Linux)

BACK

1

Rear camera (platform-generic enum value; camera capture is not supported on Linux)

Audio types

AoqAudioCaptureConfig

Field

Type

Default value

Description

is_external

bool

False

Specifies whether to use external capture mode

channel

int

1

Number of channels (mono by default)

AoqAudioPlaybackConfig

Field

Type

Default value

Description

is_external

bool

False

Specifies whether to use external playback mode

channel

int

1

Number of channels (mono by default)

AoqAudioCodecConfig

Field

Type

Default value

Description

track_type

AoqTrackType

AUDIO

Track type

codec_type

AoqEncoderType

AUDIO_PCM

Codec format

sample_rate

int

48000

Sample rate, in Hz

channel

int

1

Number of channels

bitrate

int

32000

Bitrate, in bit/s

AoqAudioFileMixConfig

Field

Type

Default value

Description

file_name

str

""

File name (including the path). It cannot be empty.

cycles

int

-1

Number of loops. -1 indicates unlimited looping.

start_pos_ms

int

0

Start playback position, in milliseconds

publish_volume

int

100

Publishing volume. Valid values: 0 to 100.

playout_volume

int

100

Playback volume. Valid values: 0 to 100.

AoqAudioExternalStreamConfig

Field

Type

Default value

Description

track_type

AoqTrackType

AUDIO

Audio track type

codec_type

AoqEncoderType

AUDIO_PCM

Audio stream format

channels

int

1

Number of channels

sample_rate

int

48000

Sample rate, in Hz

playout_volume

int

100

Playback volume. Valid values: 0 to 100.

publish_volume

int

100

Publishing volume. Valid values: 0 to 100.

max_buffer_duration

int

1000

Maximum buffer duration, in milliseconds

enable_3a

bool

False

Specifies whether to apply 3A processing to the input PCM

AoqAudioFrameData

Raw audio data, which is used for external input or observer callbacks.

Field

Type

Default value

Description

data

bytes

b""

Raw audio PCM data (a bytes copy in callbacks)

num_of_samples

int

0

Number of samples (per channel)

bytes_per_sample

int

0

Bytes per sample

num_of_channels

int

0

Number of channels

samples_per_sec

int

0

Number of samples per second (sample rate)

push_sequence

int

0

PCM input round

time_stamp

int

0

Timestamp

auto_gen_mute

bool

False

True indicates silent data generated by the SDK

AoqAudioObserverConfig

Field

Type

Default value

Description

sample_rate

int

48000

Sample rate of the callback audio, in Hz

channels

int

1

Number of audio channels in the callback

mode

AoqAudioObserverMode

READ_ONLY

Read-write mode

AoqAudioStreamDirection

Enum value

Value

Description

PUBLISH

0

Publishing stream

PLAYOUT

1

Playback stream (subscribing stream)

AoqAudioExternalStreamToggle

Enum value

Value

Description

NORMAL

0

Normal state

PAUSE

1

Paused state

AoqAudioSource

Enum value

Value

Description

CAPTURED

0

Captured audio data

PROCESS_CAPTURED

1

Audio data after 3A processing

PUBLISH

2

Audio data to be published

PLAYBACK

3

Audio data to be played

AoqAudioObserverMode

Enum value

Value

Description

READ_ONLY

0

Read-only mode

READ_WRITE

1

Read-write mode

Video types

AoqVideoCaptureConfig

Field

Type

Default value

Description

width

int

1280

Capture width, in pixels. This parameter is ineffective when is_external=True.

height

int

720

Capture height, in pixels. This parameter is ineffective when is_external=True.

fps

int

15

Capture frame rate. This parameter is ineffective when is_external=True.

is_external

bool

False

Specifies whether to use external capture. Linux supports only True. After you call start_video_capture, provide frames by calling push_external_video_frame.

camera_direction

AoqCameraDirection

FRONT

Camera direction for mobile platforms. Linux has no corresponding capability. Keep the value FRONT.

AoqVideoCodecConfig

Video codec parameters, which are shared by encoding and decoding. For decoding, only track_type/codec_type/width/height/fps/bitrate take effect, and the other fields are used only for encoding.

Field

Type

Default value

Description

track_type

AoqTrackType

VIDEO

Track type

codec_type

AoqEncoderType

VIDEO_H264

Codec format

width

int

540

Encoding width, in pixels

height

int

960

Encoding height, in pixels

fps

int

5

Encoding frame rate

bitrate

int

500000

Target bitrate, in bit/s

min_bitrate

int

128000

Minimum bitrate, in bit/s

keyframe_interval

int

2

Keyframe interval, in seconds

mirror_mode

AoqMirrorMode

DISABLED

Mirror mode

orientation_mode

AoqOrientationMode

AUTO

Video orientation mode

AoqVideoCanvas

Field

Type

Default value

Description

view

int

0

Rendering window handle. Linux has no rendering backend, so this field has no actual effect. Keep the value 0.

render_mode

AoqRenderMode

AUTO

Rendering mode. Linux has no rendering backend, so this field has no actual effect.

AoqVideoFrame

External video frame or video frame callback data. When you use a packed format (NV12/NV21/BGRA/RGBA), fill in data. When you use the three I420 planes, fill in data_y/u/v and the corresponding stride. The two methods are mutually exclusive.

Field

Type

Default value

Description

format

AoqVideoPixelFormat

UNKNOWN

Pixel format

width

int

0

Video width, in pixels

height

int

0

Video height, in pixels

data

bytes

b""

Packed-format data (NV12/NV21/BGRA/RGBA)

data_y

bytes

b""

I420 Y plane data

data_u

bytes

b""

I420 U plane data

data_v

bytes

b""

I420 V plane data

stride_y

int

0

Stride of the Y plane

stride_u

int

0

Stride of the U plane

stride_v

int

0

Stride of the V plane

time_stamp

int

0

Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock.

AoqVideoEncodedFrame

An externally encoded video frame, such as a JPEG frame.

Field

Type

Default value

Description

codec

AoqEncoderType

VIDEO_JPEG

Codec format

data

bytes

b""

Encoded data

width

int

0

Width, in pixels

height

int

0

Height, in pixels

time_stamp

int

0

Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock.

AoqVideoObserverConfig

Field

Type

Default value

Description

format

AoqVideoPixelFormat

I420

Expected pixel format of the callback data

alignment

AoqVideoObserverAlignment

DEFAULT

Width alignment policy

mode

AoqVideoObserverMode

READ_ONLY

Read-write mode

mirror_applied

bool

False

Specifies whether to mirror the callback data

AoqVideoSource

Enum value

Value

Description

CAPTURED

0

Captured video data before preprocessing

PRE_ENCODE

1

Video data before encoding, after preprocessing

REMOTE

2

Remote video data after decoding and before rendering

AoqVideoObserverMode

Enum value

Value

Description

READ_ONLY

0

Read-only mode

READ_WRITE

1

Read-write mode

AoqVideoObserverAlignment

Enum value

Value

Description

DEFAULT

0

Default alignment

EVEN

1

Even-number alignment

ALIGN_4

2

4-byte alignment

ALIGN_8

3

8-byte alignment

ALIGN_16

4

16-byte alignment

Text Generation
Image Generation
  • FAQ
Video Generation
Audio
Text Embedding
Model Production
AOQ Client SDK Linux Python API reference - Alibaba Cloud Model Studio