Skip to main content
AOQ SDK overview

AOQ Client SDK Linux C++ API reference

This topic describes the C++ APIs, callbacks, and data types of AOQ Client SDK for Linux.

API index

Engine lifecycle

API

Description

createEngine

Create the engine instance (static method, singleton)

destroy

Destroy the engine instance (static method)

getVersion

Get the SDK version (static method)

connect

Connect to the relay server

disconnect

Disconnect from the server

Audio device management

API

Description

startAudioCapture

Open the audio capture device (an empty implementation on Linux)

stopAudioCapture

Close the audio capture device (no effect on Linux)

muteAudioCapture

Mute or unmute audio capture

startAudioPlayer

Start audio rendering (empty implementation on Linux)

stopAudioPlayer

Stop audio rendering (no effect on Linux)

pauseAudioPlayer

Pause audio rendering, with fade-out supported

resumeAudioPlayer

Resume audio rendering, with fade-in supported

interruptAudioPlayer

Interrupt the current audio playback

Audio codec configuration

API

Description

setAudioEncoderConfig

Set audio encoding parameters

setAudioDecoderConfig

Set audio decoding parameters

Video device management

API

Description

startVideoCapture

Open the video capture device (only external capture mode is supported on Linux)

stopVideoCapture

Close the video capture device

setLocalView

Set or remove the local video rendering window (no rendering implementation on Linux)

setRemoteView

Set or remove the remote video rendering window (no rendering implementation on Linux)

Video codec and external input

API

Description

setVideoEncoderConfig

Set video encoding parameters

setVideoDecoderConfig

Set video decoding parameters (a codec proposal on the subscribing side)

pushExternalVideoCapturedFrame

Push an externally captured video frame

pushExternalVideoEncodedFrame

Push an externally encoded video frame

Media stream control

API

Description

enableSendMediaStream

Enable or disable sending of a local media stream

Audio file playback

API

Description

startAudioFile

Start playing a local audio file to the publishing stream

stopAudioFile

Stop audio file playback

pauseAudioFile

Pause audio file playback

resumeAudioFile

Resume audio file playback

getAudioFileDuration

Query the total duration of the audio file

getAudioFileCurrentPosition

Query the current playback position of the audio file

setAudioFilePositionMillis

Set the playback position of the audio file (seek)

setAudioFileVolume

Set the volume of the audio file

getAudioFileVolume

Query the current volume of the audio file

External audio streams

API

Description

addAudioExternalStream

Add an external audio stream

pushAudioExternalStreamData

Feed external audio PCM data

setAudioExternalStreamVolume

Set the volume of an external audio stream

getAudioExternalStreamVolume

Query the volume of an external audio stream

clearAudioExternalStreamBuffer

Clear the buffer of an external audio stream

removeAudioExternalStream

Remove an external audio stream

Real-time messaging

API

Description

sendDataMsg

Send a real-time data message

Audio frame callbacks

API

Description

setAudioFrameObserver

Register the audio frame data listener (pure virtual)

enableAudioFrameObserver

Enable or disable the audio frame callback at a specified position (pure virtual)

Local volume indication

API

Description

enableLocalAudioVolumeIndication

Enable or disable the local capture volume indication callback

Video frame callbacks

API

Description

setVideoFrameObserver

Register the video frame data listener (pure virtual)

enableVideoFrameObserver

Enable or disable the video frame callback at a specified position (pure virtual)

AoqEngineEventListener callbacks

Callback

Description

onError

Engine error callback

onWarning

Engine warning callback

onConnectionStatusChange

Connection status change callback

onStats

Engine statistics callback

onAudioDeviceStateChanged

Audio device operation status change callback

onAudioDeviceRouteChanged

Audio output route change callback (not triggered on Linux)

onAudioFileState

Audio file playback status callback

onLocalAudioVolumeIndication

Local capture volume indication callback

onVideoDeviceStateChanged

Video device operation status change callback

onDataMsg

Callback for a received real-time data message

Frame data listener interfaces

API

Description

IAudioFrameObserver

Interface for audio frame data listening (four pure virtual callbacks)

IVideoFrameObserver

Interface for video frame data listening (three pure virtual callbacks)

API details

Engine lifecycle

createEngine

Creates the engine instance. The SDK holds the engine as a global singleton, so calling this method again returns the existing instance.
static AoqClientEngine* createEngine(const AoqCreateConfig& config,
                                    AoqEngineEventListener* listener);

Parameters

Type

Description

config

AoqCreateConfig

Engine creation configuration

listener

AoqEngineEventListener\*

Engine event callback listener, which the caller implements by inheritance

Returns: A pointer to the engine instance. nullptr is returned on failure. The lifecycle of the listener must outlast that of the engine.

destroy

Destroys the engine instance and releases all resources.
static int destroy();
Returns: 0 indicates success; a non-0 value indicates a failure.

getVersion

Gets the current SDK version.
static const char* getVersion();
Returns: The version string, which is held by the SDK.

connect

Connects to the relay server. The application server obtains temporary AOQ connection parameters based on the protocol in use and sends them to the client. For more information, see Token authentication.
virtual int connect(const AoqConnectConfig& config);

Parameters

Type

Description

config

AoqConnectConfig

Connection configuration, which includes the token, SID, the array of relay access points, and the arrays of publishing and subscribing tracks

Returns: 0 indicates success and the operation runs asynchronously; non-0 indicates a failure. The connection result is notified by onConnectionStatusChange.
relayEndpoints / publishTracks / subscribeTracks are all "C-style array pointer + length" structures. The memory is held by the caller and only needs to be valid during the connect call.

disconnect

Disconnects from the server and releases the resources associated with the connection.
virtual int disconnect();
Returns: 0 indicates success; a non-0 value indicates a failure.

Audio device management

virtual int startAudioCapture(const AoqAudioCaptureConfig& config);
virtual int stopAudioCapture();
virtual int muteAudioCapture(bool mute);
virtual int startAudioPlayer(const AoqAudioPlaybackConfig& config);
virtual int stopAudioPlayer();
virtual int pauseAudioPlayer(int fadeMs);
virtual int resumeAudioPlayer(int fadeMs);
virtual int interruptAudioPlayer(AoqTrackType trackType, int fadeMs);

Parameters

Type

Description

config

AoqAudioCaptureConfig / AoqAudioPlaybackConfig

Capture and playback configuration

mute

bool

true mutes; false unmutes

fadeMs

int

Fade-out or fade-in duration, in milliseconds. 0 indicates immediate execution.

trackType

AoqTrackType

Track type

Important constraint on Linux: startAudioCapture / startAudioPlayer have empty implementations in the Linux build (they return 0 directly and do not open any sound card device), and stopAudioCapture / stopAudioPlayer have no actual effect because the internal state is not set. On Linux, use external audio streams (2.8) for audio input, and use audio frame callbacks (2.10) to play audio output yourself. For more information, see 4.3.

Audio codec configuration

virtual int setAudioEncoderConfig(const AoqAudioCodecConfig& config);
virtual int setAudioDecoderConfig(const AoqAudioCodecConfig& config);
We recommend that you call this method before connect(). An invalid combination of the sample rate and the mode triggers the onError callback with AoqECParamInvalid during connect.

Video device management

virtual int startVideoCapture(const AoqVideoCaptureConfig& config);
virtual int stopVideoCapture();
virtual int setLocalView(AoqTrackType trackType, const AoqVideoCanvas& canvas);
virtual int setRemoteView(AoqTrackType trackType, const AoqVideoCanvas& canvas);

Parameters

Type

Description

config

AoqVideoCaptureConfig

Video capture configuration

trackType

AoqTrackType

Video track type

canvas

AoqVideoCanvas

Rendering canvas. If canvas.view == nullptr, the binding is removed.

Important constraint on Linux: Camera capture on Linux is a placeholder implementation and does not produce frames. Use startVideoCapture({.isExternal = true}) + pushExternalVideoCapturedFrame to deliver frames. setLocalView / setRemoteView have no rendering backend on Linux (AoqVideoCanvas.view supports only Apple NSView* / Windows HWND / Android rendering views). For preview, render it yourself through the video frame callback (2.12). For more information, see 4.3.

Video codec and external input

virtual int setVideoEncoderConfig(const AoqVideoCodecConfig& config);
virtual int setVideoDecoderConfig(const AoqVideoCodecConfig& config);
virtual int pushExternalVideoCapturedFrame(AoqTrackType trackType, const AoqVideoFrame& frame);
virtual int pushExternalVideoEncodedFrame(AoqTrackType trackType, const AoqVideoEncodedFrame& frame);

Parameters

Type

Description

config

AoqVideoCodecConfig

Codec configuration, which is routed based on config.trackType

trackType

AoqTrackType

Routing target. Set it to AoqTrackTypeVideo.

frame

AoqVideoFrame / AoqVideoEncodedFrame

Raw frame or encoded frame data

Notes:
  • setVideoDecoderConfig is a codec proposal for the subscribing side and must be called before connect. During decoding, only trackType/codecType/width/height/fps/bitrate take effect.
  • pushExternalVideoCapturedFrame requires startVideoCapture(isExternal=true) first. It returns AoqECVideoExternalCaptureNotEnabled(211) when external capture is not started, AoqECParamInvalid when the format is not supported, and AoqECVideoExternalBufferFull(210) when the buffer is full.
  • pushExternalVideoEncodedFrame requires setVideoEncoderConfig(isExternal=true) first. The SDK packages and sends the frame directly without re-encoding. Only JPEG is currently supported.
  • When frame.timeStamp is 0, the SDK fills it in with the local time.

Media stream control

virtual int enableSendMediaStream(AoqTrackType trackType, bool enable);
Specifies whether to send a specific local media stream. trackType can be AoqTrackTypeAudio or AoqTrackTypeVideo. A return value of 0 indicates that the call is dispatched and runs asynchronously. We recommend that you disable sending after initialization and enable it only after onConnectionStatusChange reports AoqConnectionStatusConnected.

Audio file playback

virtual int startAudioFile(const char* fileId, const AoqAudioFileMixConfig& config);
virtual int stopAudioFile(const char* fileId);
virtual int pauseAudioFile(const char* fileId);
virtual int resumeAudioFile(const char* fileId);
virtual int64_t getAudioFileDuration(const char* fileId);
virtual int64_t getAudioFileCurrentPosition(const char* fileId);
virtual int setAudioFilePositionMillis(const char* fileId, int64_t positionMillis);
virtual int setAudioFileVolume(const char* fileId, AoqAudioStreamDirection type, int volume);
virtual int getAudioFileVolume(const char* fileId, AoqAudioStreamDirection type);

Parameters

Type

Description

fileId

const char\*

Audio file ID. The business layer must ensure global uniqueness.

config

AoqAudioFileMixConfig

File mixing configuration

positionMillis

int64_t

Target playback position, in milliseconds

type

AoqAudioStreamDirection

Publishing volume or local playback volume

volume

int

Volume. Valid values: 0 to 100.

Description of the return value:
  • getAudioFileDuration / getAudioFileCurrentPosition: A value >=0 is the value in milliseconds; a value <0 indicates a failure, and its absolute value is -AoqErrorCode.
  • getAudioFileVolume: A value from 0-100 is the volume; a value <0 indicates a failure, and its absolute value is -AoqErrorCode.
Playback state changes are reported through the onAudioFileState callback. File mixing uses the publishing path and does not depend on the local sound card. Therefore, on Linux, you can use it to push a local file as the audio source.

External audio streams

virtual int addAudioExternalStream(const char* streamId, const AoqAudioExternalStreamConfig& config);
virtual int pushAudioExternalStreamData(const char* streamId, AoqAudioFrameData& data);
virtual int setAudioExternalStreamVolume(const char* streamId, AoqAudioStreamDirection type, int vol);
virtual int getAudioExternalStreamVolume(const char* streamId, AoqAudioStreamDirection type);
virtual void clearAudioExternalStreamBuffer(const char* streamId, int fadeoutMs);
virtual int removeAudioExternalStream(const char* streamId);

Parameters

Type

Description

streamId

const char\*

External audio stream ID. The business layer must ensure global uniqueness.

config

AoqAudioExternalStreamConfig

External audio stream configuration

data

AoqAudioFrameData

Raw external audio data (PCM), as a non-const reference

type

AoqAudioStreamDirection

Publishing volume or local playback volume

vol

int

Volume. Valid values: 0 to 100.

fadeoutMs

int

Fade-out duration. -1 uses the SDK default fade-out, 0 clears the buffer with no fade-out, and a value greater than 0 keeps the specified fade-out duration in milliseconds.

Notes:
  • When pushAudioExternalStreamData returns AoqECAudioExternalBufferFull(110), the SDK's internal buffer is full. We recommend that you wait 20 ms and then send the current data frame again.
  • Best practice: In real-time capture scenarios, push 10 ms of data at a time and push whenever data is available. In file source scenarios, push 40 ms of data at a time at 30 ms intervals, and handle the AoqECAudioExternalBufferFull return value.
  • getAudioExternalStreamVolume: A value from 0-100 is the volume; a value <0 indicates a failure, and its absolute value is -AoqErrorCode.
  • clearAudioExternalStreamBuffer returns void and has no return value.
  • data.pushSequence is used for the SDK's consumption completion notification (PCM input round).
    There is no real capture device on Linux. This group of APIs is the main path for the audio uplink.

Real-time messaging

virtual int sendDataMsg(const AoqDataMsg& msg);
Sends a real-time data message. msg.data points to memory held by the caller and only needs to be valid during the call. Messages from the peer are reported through the onDataMsg callback.

Audio frame callbacks

virtual int setAudioFrameObserver(IAudioFrameObserver* observer) = 0;
virtual int enableAudioFrameObserver(bool enabled, AoqAudioSource audioSource,
                                     const AoqAudioObserverConfig& config) = 0;

Parameters

Type

Description

observer

IAudioFrameObserver\*

Listener instance. Pass nullptr to stop callbacks.

enabled

bool

Specifies whether to enable data callbacks at this position

audioSource

AoqAudioSource

Position of the raw audio data source

config

AoqAudioObserverConfig

Callback sample rate, number of channels, and read-write mode

Usage: First, call setAudioFrameObserver(observer) to register the listener. Then, call enableAudioFrameObserver for each position that you need. The lifecycle of the observer must cover the entire callback period. Release the observer only after you unregister it. Read-write mode support (see the IAudioFrameObserver declaration): onCapturedAudioFrame / onProcessCapturedAudioFrame / onPlaybackAudioFrame support read-write mode, whereas onPublishAudioFrame supports only read-only mode.
On Linux, setAudioFrameObserver completes registration through asynchronous dispatch (PostTask to the control thread), so the callback may take effect slightly after the call returns. Before releasing the observer, leave a sufficient safety interval or keep the object alive.

Local volume indication

virtual int enableLocalAudioVolumeIndication(const AoqAudioVolumeIndicationConfig& config);
Enables or disables local capture volume indication. If config.interval <= 0, the callback is disabled. After it is enabled, onLocalAudioVolumeIndication is triggered at the interval specified by config.interval. You must call this method after startAudioCapture to obtain volume data.
There is no device capture on Linux. The volume data in this callback comes from the publishing/playback mixing path (mixer).

Video frame callbacks

virtual int setVideoFrameObserver(IVideoFrameObserver* observer) = 0;
virtual int enableVideoFrameObserver(bool enabled, AoqVideoSource videoSource,
                                     const AoqVideoObserverConfig& config) = 0;

Parameters

Type

Description

observer

IVideoFrameObserver\*

Listener instance. Pass nullptr to stop callbacks.

enabled

bool

Specifies whether to enable data callbacks at this position

videoSource

AoqVideoSource

Data source of the video frame callback (position in the pipeline)

config

AoqVideoObserverConfig

Expected pixel format, alignment policy, read-write mode, and mirroring

If the callback returns true, the data is modified and must be written back to the SDK. This takes effect only for I420 / CVPixelBuffer, which means only I420 on Linux. If the callback returns false, the data is read-only.
Linux has no built-in rendering. This group of callbacks is the only way to obtain the remote video image (AoqVideoSourceRemote).

AoqEngineEventListener callbacks

AoqEngineEventListener is the unified entry point for all asynchronous event notifications from the SDK. You inherit and implement it, and then pass it in when you call createEngine.
Key differences from Android: All callbacks in this class are pure virtual functions (= 0) with no default empty implementation. On Linux, you must fully implement the following 10 methods; otherwise, the derived class cannot be instantiated. Callbacks may be triggered on internal SDK threads.
class AOQ_API AoqEngineEventListener {
public:
    AoqEngineEventListener();
    virtual ~AoqEngineEventListener();

    virtual void onError(int code, const char* message) = 0;
    virtual void onWarning(int code, const char* message) = 0;
    virtual void onConnectionStatusChange(AoqConnectionStatus status) = 0;
    virtual void onStats(const AoqStats& stats) = 0;
    virtual void onAudioDeviceStateChanged(const AoqAudioDeviceState& state) = 0;
    virtual void onAudioDeviceRouteChanged(int routeType) = 0;
    virtual void onAudioFileState(const AoqAudioFileState& state) = 0;
    virtual void onLocalAudioVolumeIndication(const AoqAudioVolume& volume) = 0;
    virtual void onVideoDeviceStateChanged(const AoqVideoDeviceState& state) = 0;
    virtual void onDataMsg(const AoqDataMsg& msg) = 0;
};

onError

Engine error callback. code corresponds to an AoqErrorCode enum value (see 3.3), and message is valid only during the callback.

onWarning

Engine warning callback. code corresponds to an AoqWarningCode enum value (see 3.3).

onConnectionStatusChange

Connection status change callback. State transitions: Disconnected -> Connecting -> Connected / Failed -> Disconnected.

onStats

Engine statistics callback. The SDK periodically reports publishing and subscribing statistics for audio and video and network statistics, which you can use to monitor call quality and network status in real time and to diagnose audio and video issues.

Parameters

Type

Description

stats

AoqStats

Publishing and subscribing statistics and network statistics for audio, video, and data messages

AoqStats internally uses an "array pointer + count" structure. The pointer is valid only during the callback, so you must copy the data yourself for asynchronous use.

onAudioDeviceStateChanged

Callback for audio device capture and playback operation status changes. state.reason is an AoqErrorCode value.

onAudioDeviceRouteChanged

Audio output route change callback. routeType corresponds to an AoqAudioDeviceRouteType enum value (see 3.4).
Not triggered on Linux: The Linux build does not include the platform device listener (AOQ_HAS_NATIVE_DEVICE_MONITOR is defined only on Windows / macOS), and server-side audio devices do not generate route events. You must still implement this method (pure virtual), but the implementation can be empty.

onAudioFileState

Callback for audio file playback status. state.fileId is valid only during the callback.

onLocalAudioVolumeIndication

Local capture volume indication callback. To enable it, call enableLocalAudioVolumeIndication.

onVideoDeviceStateChanged

Callback for video device capture operation status changes. state.reason is an AoqErrorCode value.

onDataMsg

Callback for a received real-time data message. msg.datais guaranteed to be valid only during the callback. If you need to use it asynchronously, copy it yourself.

Frame data listener interfaces

IAudioFrameObserver

Interface for audio data listening. Do not perform any time-consuming operations in the callbacks. Otherwise, audio anomalies may occur. All methods are pure virtual functions and must be fully implemented.
class AOQ_API IAudioFrameObserver {
public:
    virtual ~IAudioFrameObserver() {}
    virtual void onCapturedAudioFrame(const AoqAudioFrameData& data) = 0;
    virtual void onProcessCapturedAudioFrame(const AoqAudioFrameData& data) = 0;
    virtual void onPublishAudioFrame(AoqTrackType trackType, const AoqAudioFrameData& data) = 0;
    virtual void onPlaybackAudioFrame(const AoqAudioFrameData& data) = 0;
};

Callback

How to enable it (audioSource)

Read-write mode

onCapturedAudioFrame

AoqAudioSourceCaptured

Read-write supported

onProcessCapturedAudioFrame

AoqAudioSourceProcessCaptured

Read-write supported

onPublishAudioFrame

AoqAudioSourcePublish (requires a successful connect)

Only read-only is supported

onPlaybackAudioFrame

AoqAudioSourcePlayback

Read-write supported

All the preceding callbacks support setting the sample rate and the number of channels by using AoqAudioObserverConfig.
On Linux, onPlaybackAudioFrame is the main path for obtaining the remote downlink PCM. Because the 3A module is not compiled on Linux, the data from onProcessCapturedAudioFrame (data after 3A processing) is essentially the same as the raw captured data. For more information, see 4.3.

IVideoFrameObserver

Interface for video data listening. Do not perform any time-consuming operations in the callbacks. Otherwise, video stuttering may occur. All methods are pure virtual functions.
class AOQ_API IVideoFrameObserver {
public:
    virtual ~IVideoFrameObserver() {}
    virtual bool onCapturedVideoFrame(AoqVideoFrame& frame) = 0;
    virtual bool onPreEncodeVideoFrame(AoqTrackType trackType, AoqVideoFrame& frame) = 0;
    virtual bool onRemoteVideoFrame(AoqTrackType trackType, AoqVideoFrame& frame) = 0;
};

Callback

How to enable it (videoSource)

Description

onCapturedVideoFrame

AoqVideoSourceCaptured

Local raw data after capture, before preprocessing

onPreEncodeVideoFrame

AoqVideoSourcePreEncode

Local raw data before encoding, after preprocessing

onRemoteVideoFrame

AoqVideoSourceRemote

Remote raw data after decoding and before rendering

Returns: true indicates that the data is modified and must be written back to the SDK, which takes effect only for I420 / CVPixelBuffer; false indicates read-only. The pointers in frame are valid only during the callback. To use them asynchronously, copy them yourself.

Data types and enumerations

All types are defined in AoqClientEngine.h in the AoqClientSdk namespace. All structures are POD types with default values, so you get the default values in the table simply by declaring them. Fields marked as mobile-only are excluded by conditional compilation on Linux and do not exist in the structures.

General types

AoqCreateConfig

Field

Type

Default value

Description

workDir

const char\*

nullptr

SDK working directory (for logs and temporary files)

enableDumpAudio

bool

false

Specifies whether to save audio data (for debugging)

extras

const char\*

nullptr

Extended parameters (a JSON string)

String fields must remain valid at least until createEngine returns. The engine copies them internally as needed. The isBTScoMode field on Android is mobile-only and does not exist on Linux.

AoqConnectConfig

Field

Type

Default value

Description

token

const char\*

nullptr

Connection authentication token

sid

const char\*

nullptr

Session ID

certFingerprint

const char\*

nullptr

Server certificate fingerprint

workspaceIdHash

const char\*

nullptr

Workspace ID hash

relayEndpoints

const AoqRelayEndpoint\*

nullptr

Start address of the relay access point array

relayEndpointsCount

size_t

0

Number of relay access points

publishTracks

const AoqTrackParam\*

nullptr

Start address of the array of publishing track attributes

publishTracksCount

size_t

0

Number of publishing tracks

subscribeTracks

const AoqTrackParam\*

nullptr

Start address of the array of subscribing track attributes

subscribeTracksCount

size_t

0

Number of subscribing tracks

AoqRelayEndpoint

Field

Type

Default value

Description

route_index

int

-1

Route index

endpoint

const char\*

nullptr

Domain name or IP address of the relay server

port

int

0

Port of the relay server

Note that the field name route_index uses snake case, unlike the camel case used by the other fields.

AoqTrackParam

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeAudio

Track type

trackMode

AoqTrackMode

AoqTrackModeSegment

Streaming or non-streaming mode. Effective only for the audio downlink.

AoqDataMsg

Field

Type

Default value

Description

data

const uint8_t\*

nullptr

Message content. It points to memory held by the caller.

dataSize

size_t

0

Number of bytes

When the data is used in a callback, it is guaranteed to be valid only during the callback. For asynchronous use, copy the data yourself.

Statistics types

AoqStats

A summary of engine statistics, which is periodically reported through onStats. It uses an "array pointer + count" structure, in which all pointers default to nullptr and all counts default to 0.

Field

Type

Description

audioPublishStats

const AoqAudioPublishStats\*

Array of publishing statistics for audio

audioPublishStatsCount

unsigned int

Number of publishing statistics entries for audio

videoPublishStats

const AoqVideoPublishStats\*

Array of publishing statistics for video

videoPublishStatsCount

unsigned int

Number of publishing statistics entries for video

dataMsgPublishStats

const AoqDataMsgPublishStats\*

Array of publishing statistics for data messages

dataMsgPublishStatsCount

unsigned int

Number of publishing statistics entries for data messages

audioSubscribeStats

const AoqAudioSubscribeStats\*

Array of subscribing statistics for audio

audioSubscribeStatsCount

unsigned int

Number of subscribing statistics entries for audio

videoSubscribeStats

const AoqVideoSubscribeStats\*

Array of subscribing statistics for video

videoSubscribeStatsCount

unsigned int

Number of subscribing statistics entries for video

dataMsgSubscribeStats

const AoqDataMsgSubscribeStats\*

Array of subscribing statistics for data messages

dataMsgSubscribeStatsCount

unsigned int

Number of subscribing statistics entries for data messages

networkStats

const AoqNetworkStats\*

Network statistics

AoqAudioPublishStats

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeAudio

Track type

bitrate

unsigned int

0

Bitrate, in bit/s

bytes

uint64_t

0

Cumulative bytes sent

encodeVolume

unsigned int

0

Encoding volume of the publishing stream

AoqAudioSubscribeStats

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeAudio

Track type

bitrate

unsigned int

0

Bitrate, in bit/s

bytes

uint64_t

0

Cumulative bytes received

playVolume

unsigned int

0

Playback volume

AoqVideoPublishStats

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeVideo

Track type

bitrate

unsigned int

0

Bitrate, in bit/s

bytes

uint64_t

0

Cumulative bytes sent

encodeFps

unsigned int

0

Encoding frame rate

AoqVideoSubscribeStats

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeVideo

Track type

bitrate

unsigned int

0

Bitrate, in bit/s

bytes

uint64_t

0

Cumulative bytes received

decodeFps

unsigned int

0

Decoding frame rate

renderFps

unsigned int

0

Rendering frame rate

AoqDataMsgPublishStats / AoqDataMsgSubscribeStats

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeData

Track type

bitrate

unsigned int

0

Bitrate, in bit/s

bytes

uint64_t

0

Cumulative bytes sent and received

AoqNetworkStats

Field

Type

Default value

Description

sendBitrate

unsigned int

0

Send bitrate, in bit/s

sendBytes

uint64_t

0

Cumulative bytes sent

recvBitrate

unsigned int

0

Receive bitrate, in bit/s

recvBytes

uint64_t

0

Cumulative bytes received

loss

unsigned int

0

Packet loss rate, from 0 to 100

rtt

unsigned int

0

Round-trip latency, in ms

Enumerations

AoqTrackType

Enum value

Value

Description

AoqTrackTypeAudio

0

Audio track

AoqTrackTypeVideo

1

Video track

AoqTrackTypeData

2

Data message track

AoqTrackMode

Enum value

Value

Description

AoqTrackModeSegment

0

Segmented: data is packaged and delivered in semantic segments, such as a sentence.

AoqTrackModeStream

1

Streaming: data is delivered continuously.

AoqEncoderType

Enum value

Value

Description

AoqEncoderTypeUnknown

0

Unknown format

AoqEncoderTypeAudioPCM

1

Audio PCM

AoqEncoderTypeAudioOpus

2

Audio Opus (plug-in based; statically built into Linux and available out of the box)

AoqEncoderTypeVideoH264

3

Video H.264

AoqEncoderTypeVideoJpeg

4

Video JPEG

AoqEncoderTypeDataText

5

Message text

AoqConnectionStatus

Enum value

Value

Description

AoqConnectionStatusDisconnected

0

Disconnected

AoqConnectionStatusConnecting

1

Connecting

AoqConnectionStatusConnected

2

Connected

AoqConnectionStatusFailed

3

Connection failed

AoqErrorCode

Naming note: In C++ headers, the enum literals use the abbreviated AoqEC* form (unlike the full AoqErrorCode* form on Android), but the values are identical.

Enum value

Value

Description

AoqECOK

0

Success

AoqECParamInvalid

1

Invalid parameter

AoqECStateInvalid

2

Invalid state

AoqECUnSupport

3

The API is called but is not implemented on the current platform or in the current mode

AoqECAudio

100

Generic audio error

AoqECAudioExternalBufferFull

110

External audio buffer is full

AoqECAudioDevice

120

Generic audio device error

AoqECAudioDeviceRecordingAuthFailed

121

Recording permission not granted

AoqECAudioDeviceRecordingOccupied

122

Recording device is in use

AoqECAudioDeviceRecordingBackgroundStart

123

Failed to start recording in the background

AoqECAudioDeviceRecordingStartFail

124

Failed to start recording

AoqECAudioDevicePlayoutOccupied

125

Playback device is in use

AoqECAudioDevicePlayoutBackgroundStart

126

Failed to start playback in the background

AoqECAudioDevicePlayoutStartFail

127

Failed to start playback

AoqECAudioDeviceEarpieceRequiresVoipMode

128

The receiver requires VoIP mode (mobile scenarios)

AoqECVideo

200

Generic video error

AoqECVideoExternalBufferFull

210

External video buffer is full

AoqECVideoExternalCaptureNotEnabled

211

External video capture is not enabled

AoqECVideoExternalEncoderNotEnabled

212

External video encoding is not enabled

AoqECVideoDevice

220

Generic video device error

AoqECVideoDeviceCameraOpenFail

221

Failed to open the camera

AoqECVideoDeviceCameraAuthFailed

222

Camera permission not granted

AoqECVideoDeviceCameraOccupied

223

Camera is in use

AoqECVideoDeviceCameraRunningError

224

Camera runtime exception

AoqECVideoCodec

230

Generic video codec error

AoqECVideoCodecEncoderInitFail

231

Failed to initialize the video encoder

AoqECVideoRender

240

Generic video rendering error

AoqECVideoRenderCreateFail

241

Failed to create video rendering

AoqECVideoRenderDrawError

242

Video rendering drawing error

AoqWarningCode

Enum value

Value

Description

AoqWCOK

0

No warning

AoqWCAudio

100

Generic audio warning

AoqWCAudioHowling

101

Audio howling detected

AoqWCAudioDevice

120

Generic audio device warning

AoqWCAudioDeviceMicEnumerateError

121

Microphone enumeration error

AoqWCAudioDeviceMicStartTimeout

122

Microphone startup timed out

AoqWCAudioDeviceRecordingError

123

Error during recording

AoqWCAudioDeviceSpeakerEnumerateError

124

Speaker enumeration error

AoqWCAudioDeviceSpeakerStartTimeout

125

Speaker startup timed out

AoqWCAudioDevicePlayoutError

126

Error during playback

AoqWCVideo

200

Generic video warning

AoqWCVideoCameraEnumerateError

201

Camera enumeration error

AoqWCVideoEncoderSwitched

202

Video encoder switched

AoqWCVideoRenderDowngrade

203

Video rendering downgraded

AoqMirrorMode

Enum value

Value

Description

AoqMirrorModeDisabled

0

Disable mirroring

AoqMirrorModeEnabled

1

Enable mirroring

AoqOrientationMode

Enum value

Value

Description

AoqOrientationModeAuto

0

Auto-fit

AoqOrientationModePortrait

1

Portrait

AoqOrientationModeLandscape

2

Landscape

AoqRenderMode

Enum value

Value

Description

AoqRenderModeAuto

0

Adaptive mode

AoqRenderModeStretch

1

Stretch mode

AoqRenderModeFill

2

Fill mode

AoqRenderModeCrop

3

Crop mode

This is used only as AoqVideoCanvas.renderMode. Linux has no rendering implementation, so this enumeration does not actually take effect.

AoqAudioStreamDirection

Enum value

Value

Description

AoqAudioStreamPublish

0

Publishing audio

AoqAudioStreamPlayout

1

Playback audio

AoqAudioExternalStreamToggle

Enum value

Value

Description

AoqAudioExternalStreamToggleNormal

0

Recovered

AoqAudioExternalStreamTogglePause

1

Paused

This enumeration is declared in the header, but no public API currently uses it. It is a reserved definition.

Audio types

AoqAudioCaptureConfig

Field

Type

Default value

Description

isExternal

bool

false

Specifies whether to use external capture

channel

int

1

Number of audio capture channels. 1 and 2 are supported.

isVoipMode is a mobile-only field and does not exist on Linux.

AoqAudioPlaybackConfig

Field

Type

Default value

Description

isExternal

bool

false

Specifies whether to use external playback

channel

int

1

Number of audio playback channels. 1 and 2 are supported.

isVoipMode / isDefaultSpeaker are mobile-only fields and do not exist on Linux.

AoqAudioCodecConfig

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeAudio

Track type

codecType

AoqEncoderType

AoqEncoderTypeAudioPCM

Encoding type

sampleRate

int

48000

Sample rate, in Hz

channel

int

1

Number of channels. 1 and 2 are supported.

bitrate

int

32000

Bitrate, in bit/s

Sample rate constraints (excerpted from the comments in the header file):
  • Encoding: Opus supports 8 / 16 / 48K, and PCM supports 8 / 16 / 32 / 48K;
  • Decoding: 24K is additionally supported, but 24K is limited to Segment mode (Stream mode does not support 24K);
  • An invalid combination triggers the onError callback with AoqECParamInvalid during connect.

AoqAudioDeviceRouteType

Enum value

Value

Description

AoqAudioDeviceRouteDefault

0

Default route

AoqAudioDeviceRouteHeadset

1

Headphones

AoqAudioDeviceRouteEarpiece

2

Receiver

AoqAudioDeviceRouteHeadsetNoMic

3

Headphones without a microphone

AoqAudioDeviceRouteSpeakerPhone

4

Built-in speaker

AoqAudioDeviceRouteUsb

5

USB audio device

AoqAudioDeviceRouteBluetooth

6

Bluetooth

AoqAudioDeviceRouteBluetoothA2dp

7

Bluetooth A2DP

This is only a reference for the routeType values of onAudioDeviceRouteChanged. Linux does not trigger this callback.

AoqAudioDeviceStateCode

Enum value

Value

Description

AoqAudioDeviceNone

0

No state

AoqAudioDeviceRecordStarting

1

Capture starting

AoqAudioDeviceRecordStarted

2

Capture started

AoqAudioDeviceRecordStopping

3

Capture stopping

AoqAudioDeviceRecordStopped

4

Capture stopped

AoqAudioDeviceRecordFail

5

Capture failed

AoqAudioDevicePlayStarting

6

Playback starting

AoqAudioDevicePlayStarted

7

Playback started

AoqAudioDevicePlayStopping

8

Playback stopping

AoqAudioDevicePlayStopped

9

Playback stopped

AoqAudioDevicePlayFail

10

Playback failed

AoqAudioDeviceState

Field

Type

Default value

Description

state

AoqAudioDeviceStateCode

AoqAudioDeviceNone

Device operation status

reason

int

0

Error reason code. See AoqErrorCode.

AoqAudioFrameData

Raw audio data. It is used for both external stream feeding through pushAudioExternalStreamData and the IAudioFrameObserver callback.

Field

Type

Default value

Description

dataPtr

void\*

0

Pointer to the audio PCM data

numOfSamples

int

0

Number of samples (per channel)

bytesPerSample

int

0

Bytes per sample

numOfChannels

int

0

Number of channels

samplesPerSec

int

0

Number of samples per second (sample rate)

pushSequence

int

0

PCM input round (used for the stream consumption completion notification)

timeStamp

int64_t

0

Timestamp

dataSize

int

0

Data length, in bytes

autoGenMute

bool

false

Effective only in callbacks. true indicates silent data generated by the SDK.

AoqAudioSource

Enum value

Value

Description

AoqAudioSourceCaptured

0

Captured audio data

AoqAudioSourceProcessCaptured

1

Audio data after 3A processing

AoqAudioSourcePublish

2

Audio data to be published (requires a successful connect)

AoqAudioSourcePlayback

3

Audio data to be played

AoqAudioSourceMax

4

Placeholder, do not use

AoqAudioObserverMode

Enum value

Value

Description

AoqAudioObserverModeReadOnly

0

Read-only mode

AoqAudioObserverModeReadWrite

1

Read-write mode

AoqAudioObserverConfig

Field

Type

Default value

Description

sampleRate

int

48000

Sample rate of the callback audio, in Hz. Resampling is performed if the rate does not match that of the source.

channels

int

1

Number of audio channels in the callback. 1 and 2 are supported. This parameter is limited by the codec parameters of the subscribing stream.

mode

AoqAudioObserverMode

AoqAudioObserverModeReadOnly

Callback mode

Supported sample rates: 8, 12, 16, 24, 32, 44.1, 48, 64, 88.2, 96, 176.4, and 192K.

AoqAudioVolumeIndicationConfig

Field

Type

Default value

Description

interval

int

0

Callback interval, in milliseconds. A value less than or equal to 0 disables the callback. A value greater than 0 and less than 10 is treated as 10.

smooth

int

3

Volume smoothing coefficient. A larger value results in smoother output. Valid values: 0 to 10.

AoqAudioVolume

Field

Type

Default value

Description

volume

int

0

Smoothed instantaneous volume. Valid values: 0 to 255.

Audio file and external audio stream types

AoqAudioFileMixConfig

Field

Type

Default value

Description

fileName

const char\*

nullptr

File name (including the path)

cycles

int

-1

Number of loops. -1 indicates unlimited looping.

startPosMs

long

0

Start playback position, in milliseconds

publishVolume

int

100

Publishing volume. Valid values: 0 to 100.

playoutVolume

int

100

Playback volume. Valid values: 0 to 100.

fileId is not part of the configuration body and is passed as a separate parameter to startAudioFile.

AoqAudioFileStateCode

Enum value

Value

Description

AoqAudioFileNone

0

No state

AoqAudioFileStarted

1

Started

AoqAudioFileStopped

2

Stopped

AoqAudioFilePaused

3

Paused

AoqAudioFileResumed

4

Resumed

AoqAudioFileEnded

5

Playback completed

AoqAudioFileBuffering

6

Buffering

AoqAudioFileBufferingEnd

7

Buffering ended

AoqAudioFileFailed

8

Playback failed

AoqAudioFileErrorCode

Enum value

Value

Description

AoqAudioFileNoError

0

No error

AoqAudioFileOpenFailed

1

Failed to open the file

AoqAudioFileDecodeFailed

2

Failed to decode the file

AoqAudioFileState

Field

Type

Default value

Description

fileId

const char\*

nullptr

File ID. It is valid only during the callback.

stateCode

AoqAudioFileStateCode

AoqAudioFileNone

File status code

errorCode

AoqAudioFileErrorCode

AoqAudioFileNoError

File error code

AoqAudioExternalStreamConfig

Field

Type

Default value

Description

trackType

AoqTrackType

AoqTrackTypeAudio

Audio track type

codecType

AoqEncoderType

AoqEncoderTypeAudioPCM

Audio stream format. PCM is supported.

channels

int

1

Number of channels. It is limited by the audio codec of the publishing stream (1 and 2 are supported, and additional channels are ignored).

sampleRate

int

48000

Sample rate, in Hz

playoutVolume

int

100

Playback volume. Valid values: 0 to 100.

publishVolume

int

100

Publishing volume. Valid values: 0 to 100.

maxBufferDuration

int

600000

Maximum buffer duration, in milliseconds (that is, 10 minutes). Valid values: 100 and above. If the duration exceeds this value, Push fails.

enable3A

bool

false

Apply 3A processing to the input PCM (does not take effect on Linux, see 4.3)

Supported sample rates: 8, 12, 16, 24, 32, 44.1, 48, 64, 88.2, 96, 176.4, and 192K.
streamId is not part of the configuration body and is passed as a separate parameter to addAudioExternalStream.

Video types

AoqVideoCanvas

Field

Type

Default value

Description

view

void\*

nullptr

An opaque pointer to the platform-native view handle. Pass nullptr to remove the binding.

renderMode

AoqRenderMode

AoqRenderModeAuto

Rendering fill mode

Platform meanings of view: NSView* on Apple, HWND on Windows, and the SDK's internal rendering view adapter object on Android. You must ensure that the lifecycle of view is longer than the period during which it is set by setLocalView / setRemoteView.
No corresponding platform implementation on Linux. This structure has no practical use on Linux.

AoqVideoCaptureConfig

Field

Type

Default value

Description

width

int

1280

Capture width, in pixels. This parameter is ineffective when isExternal=true.

height

int

720

Capture height, in pixels. This parameter is ineffective when isExternal=true.

fps

int

15

Capture frame rate. This parameter is ineffective when isExternal=true (the pace is determined by frame delivery).

isExternal

bool

false

Specifies whether to use external capture. If it is true, the camera is not opened, and frames are fed by pushExternalVideoCapturedFrame.

cameraDirection is a mobile-only field and does not exist on desktop platforms (including Linux). On Linux, we recommend that you always set isExternal = true.

AoqVideoCodecConfig

Encoding and decoding share the same structure. For decoding, only trackType / codecType / width / height / fps / bitrate take effect and participate in negotiation as a subscription proposal. minBitrate / keyframeInterval / mirrorMode / orientationMode / isExternal are used only for encoding.

Field

Type

Default value

Description

isExternal

bool

false

External encoding mode. When it is true, you push encoded frames, and the SDK does not perform capture or encoding.

trackType

AoqTrackType

AoqTrackTypeVideo

Track type

codecType

AoqEncoderType

AoqEncoderTypeVideoH264

Encoding type

width

int

540

Resolution width, in pixels

height

int

960

Resolution height, in pixels

fps

int

5

Frame rate

bitrate

int

500000

Initial bitrate, in bit/s

minBitrate

int

128000

Minimum bitrate, in bit/s

keyframeInterval

int

2

Keyframe interval, in seconds

mirrorMode

AoqMirrorMode

AoqMirrorModeDisabled

Mirror mode

orientationMode

AoqOrientationMode

AoqOrientationModeAuto

Orientation mode

AoqVideoPixelFormat

Enum value

Value

Description

Supported on Linux

AoqVideoPixelFormatUnknown

0

Unknown format

-

AoqVideoPixelFormatI420

1

I420 (YUV planar format)

Supported

AoqVideoPixelFormatNV12

2

NV12 (YUV semi-planar format)

Supported

AoqVideoPixelFormatNV21

3

NV21 (YUV semi-planar format)

Supported

AoqVideoPixelFormatBGRA

4

BGRA (32-bit)

Supported

AoqVideoPixelFormatRGBA

5

RGBA (32-bit)

Supported

AoqVideoPixelFormatCVPixelBuffer

6

Effective only on Apple platforms

Not supported

AoqVideoPixelFormatTextureOES

7

Android external OES texture

Not supported

AoqVideoPixelFormatTexture2D

8

Android regular 2D texture

Not supported

Enum values 6/7/8 are still visible in the header file (they are not excluded by conditional compilation), but Linux does not have the corresponding platform capability. If you pass them, they are treated as unsupported formats.

AoqVideoFrame

An external raw video frame. It is used for both pushExternalVideoCapturedFrame and the IVideoFrameObserver callback.

Field

Type

Default value

Description

format

AoqVideoPixelFormat

AoqVideoPixelFormatUnknown

Pixel format

width

int

0

Width, in pixels

height

int

0

Height, in pixels

dataPtr

void\*

0

Packed-format data (NV12 / NV21 / BGRA / RGBA)

dataSize

int

0

Number of bytes of packed data

dataY

void\*

0

I420 Y plane

dataU

void\*

0

I420 U plane

dataV

void\*

0

I420 V plane

strideY

int

0

Stride of the Y plane

strideU

int

0

Stride of the U plane

strideV

int

0

Stride of the V plane

nativePixelBuffer

void\*

0

Used for Apple zero-copy (CVPixelBufferRef). This field is not used on Linux.

textureId

int

0

Android texture ID. This field is not used on Linux.

transformMatrix

float array (16 elements)

All zeros

Texture transformation matrix (row-major 4x4). This field is not used on Linux.

timeStamp

int64_t

0

Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local time.

AoqVideoCodecType

Enum value

Value

Description

AoqVideoCodecTypeJPEG

0

JPEG encoding

AoqVideoEncodedFrame

An externally encoded video frame. The caller performs the encoding, and the SDK packages and sends the frame directly without re-encoding.

Field

Type

Default value

Description

codec

AoqVideoCodecType

AoqVideoCodecTypeJPEG

Codec format

data

void\*

nullptr

Pointer to the encoded data

dataSize

int

0

Number of bytes of the encoded data

width

int

0

Width, in pixels

height

int

0

Height, in pixels

timeStamp

int64_t

0

Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local time.

AoqVideoDeviceStateCode

Enum value

Value

Description

AoqVideoDeviceNone

0

No state

AoqVideoDeviceCaptureStarting

1

Camera starting

AoqVideoDeviceCaptureStarted

2

Camera started

AoqVideoDeviceCaptureStopping

3

Camera stopping

AoqVideoDeviceCaptureStopped

4

Camera stopped

AoqVideoDeviceCaptureFail

5

Failed to start the camera (for example, permission denied or the device is unavailable)

AoqVideoDeviceState

Field

Type

Default value

Description

state

AoqVideoDeviceStateCode

AoqVideoDeviceNone

Device capture operation status

reason

int

0

Error reason code. See AoqErrorCode.

AoqVideoSource

Enum value

Value

Description

AoqVideoSourceCaptured

0

Captured video data before preprocessing

AoqVideoSourcePreEncode

1

Video data before encoding, after preprocessing

AoqVideoSourceRemote

2

Remote video data after decoding and before rendering

AoqVideoSourceMax

3

Placeholder, do not use

AoqVideoObserverMode

Enum value

Value

Description

AoqVideoObserverModeReadOnly

0

Read-only mode

AoqVideoObserverModeReadWrite

1

Read-write mode (supported only for I420 / CVPixelBuffer)

AoqVideoObserverAlignment

Enum value

Value

Description

AoqVideoObserverAlignmentDefault

0

Default alignment

AoqVideoObserverAlignmentEven

1

Even-number alignment

AoqVideoObserverAlignment4

2

4-byte alignment

AoqVideoObserverAlignment8

3

8-byte alignment

AoqVideoObserverAlignment16

4

16-byte alignment

AoqVideoObserverConfig

Field

Type

Default value

Description

format

AoqVideoPixelFormat

AoqVideoPixelFormatI420

Expected pixel format of the callback data

alignment

AoqVideoObserverAlignment

AoqVideoObserverAlignmentDefault

Width alignment policy

mode

AoqVideoObserverMode

AoqVideoObserverModeReadOnly

Callback mode. ReadWrite is supported only for I420 / CVPixelBuffer.

mirrorApplied

bool

false

Specifies whether to mirror the callback data

Text Generation
Image Generation
  • FAQ
Video Generation
Audio
Text Embedding
Model Production