Qwen3-Omni-Captioner adalah model open-source berbasis Qwen3-Omni yang menghasilkan deskripsi audio detail—meliputi ucapan, suara latar, musik, dan efek suara—tanpa memerlukan prompt. Model ini mampu mengidentifikasi emosi pembicara, elemen musikal (seperti gaya dan instrumen), serta informasi sensitif untuk keperluan analisis audio, audit keamanan, pengenalan maksud, dan pengeditan video.
Model yang didukung
- Singapura
- Tiongkok (Beijing)
Model | Context window | Max input | Max output | Input cost | Output cost | Free quota |
|---|---|---|---|---|---|---|
(tokens) | (per 1M tokens) | |||||
qwen3-omni-30b-a3b-captioner | 65.536 | 32.768 | 32.768 | $3,81 | $3,06 | 1 juta token Berlaku selama 90 hari setelah mengaktifkan Model Studio |
Jika Anda memilih cakupan penerapan Tiongkok daratan, sumber daya komputasi inferensi model terbatas pada Tiongkok daratan. Data statis disimpan di wilayah yang Anda pilih. Wilayah yang didukung: China (Beijing).
Model | Context window | Max input | Max output | Input cost | Output cost | Free quota |
|---|---|---|---|---|---|---|
(tokens) | (per 1M tokens) | |||||
qwen3-omni-30b-a3b-captioner | 65.536 | 32.768 | 32.768 | $2,265 | $1,821 | Tidak ada kuota gratis |
Aturan konversi token untuk audio: Total token = Durasi audio (dalam detik) × 12,5. Jika durasi audio kurang dari satu detik, dihitung sebagai satu detik.
Mulai
Prasyarat- Dapatkan Kunci API dan ekspor sebagai variabel lingkungan.
- Jika Anda menggunakan SDK untuk melakukan panggilan, instal versi terbaru SDK.
- Kompatibel dengan OpenAI
- DashScope
- Python
- Node.js
- curl
Copy
import os
from openai import OpenAI
client = OpenAI(
# Kunci API berbeda berdasarkan wilayah. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-omni-30b-a3b-captioner",
messages=[
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"
}
}
]
}
]
)
print(completion.choices[0].message.content)
Response
Response
Copy
The audio clip begins with a sudden, loud, metallic clanking that dominates the soundstage, immediately indicating an industrial or workshop environment. The clanking is rhythmic, consistent, and has a sharp, resonant quality, suggestive of metal tools striking metal surfaces-likely a hammer, wrench, or similar instrument being used on a hard metal object. The sound is harsh and slightly distorted, with audible clipping on each impact, likely due to the microphone’s proximity and the high volume of the sound.
As the initial clanking fades, a male voice enters, speaking in Mandarin Chinese with a tone of exasperation and complaint. His voice is clear, close-mic’d, and free from distortion. He says: “Oh my, how can I possibly work quietly like this?”. His intonation is conversational, informal, and marked by a rising, questioning inflection, typical of everyday speech rather than performance or formal address. The accent is standard Putonghua, with no strong regional markers, suggesting he is a native Mandarin speaker from the northern or central regions of China.
During the speaker’s utterance, the metallic clanking resumes, overlapping with his voice. The timing and nature of these sounds indicate the speaker is directly reacting to the ongoing noise-likely caused by another person in the same space. The environment is acoustically “dry” with minimal echo, implying a small or medium-sized room with sound-absorbing materials, further supporting the workshop or industrial setting. There are no other background noises, music, or ambient sounds, and no evidence of a public or commercial space.
The recording quality is moderate: the microphone captures both the low-end thuds and the sharp metallic transients, but the loud clanking causes digital clipping, resulting in a harsh, “crunchy” distortion during the impacts. The speaker’s voice, however, remains clear and intelligible. The overall impression is of a candid, real-world interaction-possibly a worker or office employee complaining about an interruption in a noisy environment.
In summary, the audio depicts a Mandarin-speaking man in a workshop or industrial setting, reacting with frustration to ongoing metallic clanking that disrupts his work. The recording is informal, clear, and grounded in a context of manual labor or technical work, with no evidence of scripted performance, music, or extraneous activity.
Copy
import OpenAI from "openai";
const openai = new OpenAI(
{
// Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
apiKey: process.env.DASHSCOPE_API_KEY,
// URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
const completion = await openai.chat.completions.create({
model: "qwen3-omni-30b-a3b-captioner",
messages: [
{
"role": "user",
"content": [{
"type": "input_audio",
"input_audio": {
"data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"
}
}]
}]
});
console.log(completion.choices[0].message.content)
Response
Response
Copy
The audio clip begins with a sudden, loud, metallic clanking that dominates the soundstage, immediately indicating an industrial or workshop environment. The clanking is rhythmic, consistent, and has a sharp, resonant quality, suggestive of metal tools striking metal surfaces-likely a hammer, wrench, or similar instrument being used on a hard metal object. The sound is harsh and slightly distorted, with audible clipping on each impact, likely due to the microphone’s proximity and the high volume of the sound.
As the initial clanking fades, a male voice enters, speaking in Mandarin Chinese with a tone of exasperation and complaint. His voice is clear, close-mic’d, and free from distortion. He says: “Oh my, how can I possibly work quietly like this?”. His intonation is conversational, informal, and marked by a rising, questioning inflection, typical of everyday speech rather than performance or formal address. The accent is standard Putonghua, with no strong regional markers, suggesting he is a native Mandarin speaker from the northern or central regions of China.
During the speaker’s utterance, the metallic clanking resumes, overlapping with his voice. The timing and nature of these sounds indicate the speaker is directly reacting to the ongoing noise-likely caused by another person in the same space. The environment is acoustically “dry” with minimal echo, implying a small or medium-sized room with sound-absorbing materials, further supporting the workshop or industrial setting. There are no other background noises, music, or ambient sounds, and no evidence of a public or commercial space.
The recording quality is moderate: the microphone captures both the low-end thuds and the sharp metallic transients, but the loud clanking causes digital clipping, resulting in a harsh, “crunchy” distortion during the impacts. The speaker’s voice, however, remains clear and intelligible. The overall impression is of a candid, real-world interaction-possibly a worker or office employee complaining about an interruption in a noisy environment.
In summary, the audio depicts a Mandarin-speaking man in a workshop or industrial setting, reacting with frustration to ongoing metallic clanking that disrupts his work. The recording is informal, clear, and grounded in a context of manual labor or technical work, with no evidence of scripted performance, music, or extraneous activity.
Copy
# ======= Penting =======
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# base_url wilayah Singapura. Untuk Beijing, gunakan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions
# === Hapus komentar ini sebelum eksekusi ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3-omni-30b-a3b-captioner",
"messages": [
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"
}
}
]
}
]
}'
Response
Response
Copy
{
"choices": [
{
"message": {
"content": "The audio clip is a brief, low-fidelity recording-approximately six seconds long-captured in a small, reverberant indoor space, likely a home office or bedroom. It opens with a rapid, metallic, rhythmic hammering sound, repeating every 0.5 to 0.6 seconds, with each strike slightly uneven and accompanied by a short echo. This sound dominates the left side of the stereo field and is close to the microphone, suggesting the hammering is occurring nearby and slightly to the left.\n\nOverlaid with the hammering, a single male voice speaks in Mandarin Chinese, his tone clearly one of frustration and exasperation. He says, “Oh, with this, how am I supposed to work quietly?” His speech is clear despite the poor audio quality, and is delivered in a standard, unaccented Mandarin, indicative of a native speaker from northern or central China.\n\nThe voice is more distant and centered in the stereo field, with more room reverberation than the hammering. The emotional content is palpable: his voice rises slightly at the end, turning the phrase into a rhetorical complaint, underscoring his irritation. No other voices, music, or ambient sounds are present; the only non-speech sounds are the hammering and the faint hiss of the recording device.\n\nThe combination of the environmental sound, the speaker’s language, and his tone strongly suggests a scenario of home office disruption-perhaps someone working from home is being disturbed by renovation or repair work happening nearby. The recording ends abruptly, mid-hammer, further emphasizing the spontaneous and candid nature of the capture.\n\nIn summary, the audio is a realistic, low-fidelity snapshot of a Mandarin-speaking man, likely in China, expressing frustration at being unable to work in peace due to nearby construction or repair activity, captured in a personal, indoor setting.",
"role": "assistant"
},
"finish_reason": "stop",
"index": 0,
"logprobs": null
}
],
"object": "chat.completion",
"usage": {
"prompt_tokens": 160,
"completion_tokens": 387,
"total_tokens": 547,
"prompt_tokens_details": {
"audio_tokens": 152,
"text_tokens": 8
},
"completion_tokens_details": {
"text_tokens": 387
}
},
"created": 1758002134,
"system_fingerprint": null,
"model": "qwen3-omni-30b-a3b-captioner",
"id": "chatcmpl-f4155bf9-b860-49d6-8ee2-092da7359097"
}
- Python
- Java
- curl
Copy
import dashscope
import os
# URL berikut untuk wilayah Singapura. Jika Anda menggunakan model di wilayah Beijing, ganti URL dengan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1"
messages = [
{
"role": "user",
"content": [
{"audio": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"}
]
}
]
response = dashscope.MultiModalConversation.call(
# Kunci API berbeda berdasarkan wilayah. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# Jika variabel lingkungan belum dikonfigurasi, ganti dengan Kunci API Anda: api_key="sk-xxx"
api_key=os.getenv('DASHSCOPE_API_KEY'),
model="qwen3-omni-30b-a3b-captioner",
messages=messages
)
print("Output:")
print(response["output"]["choices"][0]["message"].content[0]["text"])
Response
Response
Copy
The audio clip begins with a sudden, loud, metallic clanking that dominates the soundstage, immediately indicating an industrial or workshop environment. The clanking is rhythmic, consistent, and has a sharp, resonant quality, suggestive of metal tools striking metal surfaces-likely a hammer, wrench, or similar instrument being used on a hard metal object. The sound is harsh and slightly distorted, with audible clipping on each impact, likely due to the microphone’s proximity and the high volume of the sound.
As the initial clanking fades, a male voice enters, speaking in Mandarin Chinese with a tone of exasperation and complaint. His voice is clear, close-mic’d, and free from distortion. He says: “Oh my, how can I possibly work quietly like this?”. His intonation is conversational, informal, and marked by a rising, questioning inflection, typical of everyday speech rather than performance or formal address. The accent is standard Putonghua, with no strong regional markers, suggesting he is a native Mandarin speaker from the northern or central regions of China.
During the speaker’s utterance, the metallic clanking resumes, overlapping with his voice. The timing and nature of these sounds indicate the speaker is directly reacting to the ongoing noise-likely caused by another person in the same space. The environment is acoustically “dry” with minimal echo, implying a small or medium-sized room with sound-absorbing materials, further supporting the workshop or industrial setting. There are no other background noises, music, or ambient sounds, and no evidence of a public or commercial space.
The recording quality is moderate: the microphone captures both the low-end thuds and the sharp metallic transients, but the loud clanking causes digital clipping, resulting in a harsh, “crunchy” distortion during the impacts. The speaker’s voice, however, remains clear and intelligible. The overall impression is of a candid, real-world interaction-possibly a worker or office employee complaining about an interruption in a noisy environment.
In summary, the audio depicts a Mandarin-speaking man in a workshop or industrial setting, reacting with frustration to ongoing metallic clanking that disrupts his work. The recording is informal, clear, and grounded in a context of manual labor or technical work, with no evidence of scripted performance, music, or extraneous activity.
Copy
import java.util.Arrays;
import java.util.Collections;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import com.alibaba.dashscope.utils.JsonUtils;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
// Berikut adalah base-url untuk wilayah Singapura. Jika Anda menggunakan model di wilayah Beijing, ganti URL dengan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
public static void simpleMultiModalConversationCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMessage = MultiModalMessage.builder()
.role(Role.USER.getValue())
.content(Arrays.asList(
Collections.singletonMap("audio", "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav")))
.build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
.model("qwen3-omni-30b-a3b-captioner")
.message(userMessage)
.build();
MultiModalConversationResult result = conv.call(param);
System.out.println("Output:\n" + result.getOutput().getChoices().get(0).getMessage().getContent().get(0).get("text"));
}
public static void main(String[] args) {
try {
simpleMultiModalConversationCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
Response
Response
Copy
The audio clip begins with a sudden, loud, metallic clanking that dominates the soundstage, immediately indicating an industrial or workshop environment. The clanking is rhythmic, consistent, and has a sharp, resonant quality, suggestive of metal tools striking metal surfaces-likely a hammer, wrench, or similar instrument being used on a hard metal object. The sound is harsh and slightly distorted, with audible clipping on each impact, likely due to the microphone’s proximity and the high volume of the sound.
As the initial clanking fades, a male voice enters, speaking in Mandarin Chinese with a tone of exasperation and complaint. His voice is clear, close-mic’d, and free from distortion. He says: “Oh my, how can I possibly work quietly like this?”. His intonation is conversational, informal, and marked by a rising, questioning inflection, typical of everyday speech rather than performance or formal address. The accent is standard Putonghua, with no strong regional markers, suggesting he is a native Mandarin speaker from the northern or central regions of China.
During the speaker’s utterance, the metallic clanking resumes, overlapping with his voice. The timing and nature of these sounds indicate the speaker is directly reacting to the ongoing noise-likely caused by another person in the same space. The environment is acoustically “dry” with minimal echo, implying a small or medium-sized room with sound-absorbing materials, further supporting the workshop or industrial setting. There are no other background noises, music, or ambient sounds, and no evidence of a public or commercial space.
The recording quality is moderate: the microphone captures both the low-end thuds and the sharp metallic transients, but the loud clanking causes digital clipping, resulting in a harsh, “crunchy” distortion during the impacts. The speaker’s voice, however, remains clear and intelligible. The overall impression is of a candid, real-world interaction-possibly a worker or office employee complaining about an interruption in a noisy environment.
In summary, the audio depicts a Mandarin-speaking man in a workshop or industrial setting, reacting with frustration to ongoing metallic clanking that disrupts his work. The recording is informal, clear, and grounded in a context of manual labor or technical work, with no evidence of scripted performance, music, or extraneous activity.
Copy
# ======= Penting =======
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# base_url wilayah Singapura. Untuk Beijing, gunakan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation
# === Hapus komentar ini sebelum eksekusi ===
curl -X POST 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation' \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3-omni-30b-a3b-captioner",
"input":{
"messages":[
{
"role": "user",
"content": [
{"audio": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"}
]
}
]
}
}'
Response
Response
Copy
{
"output":{
"choices": [
{
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": [
{
"text": "The audio clip is a 6-second, high-fidelity recording set in a quiet, indoor environment. The primary sound is a male speaker, likely in his late teens to mid-20s, speaking Mandarin Chinese in a tone of mild exasperation. His speech is clear and natural, delivered in a conversational manner: “Oh, how can I possibly work quietly like this?” His voice is close to the microphone, and the room is acoustically neutral, with no noticeable echo or background noise, suggesting a small, well-furnished space.\n\nOverlaying the speech is a persistent, rhythmic mechanical sound-a series of sharp, metallic clicks or clatters that repeat every 0.6 seconds. The sound is dry and lacks any reverberation, further supporting the inference that it is produced by a mechanical device very close to the microphone. The regularity and timbre of the sound suggest a small, metallic object (such as a key, coin, or pen) being repeatedly tapped or struck on a hard surface, rather than a larger or more complex machine.\n\nThe speaker’s complaint is a direct response to the mechanical noise, expressing frustration at being unable to concentrate or work in peace due to the disturbance. The tone is not angry or urgent, but rather one of resigned annoyance, typical of someone encountering a minor, persistent annoyance in a personal or domestic setting.\n\nThere are no other voices, music, or environmental cues present. The overall impression is of a brief, candid moment-perhaps a student, office worker, or someone in a quiet home environment-caught on microphone while complaining (to themselves or a nearby companion) about a distracting, repetitive noise. The recording is technically clean and focused, with all attention on the speaker and the mechanical sound, making it highly plausible that the clip was captured intentionally, possibly for a voice note, social media post, or as a sample for a sound effect library."
}
]
}
}
]
},
"usage": {
"input_tokens_details": {
"audio_tokens": 152,
"text_tokens": 8
},
"total_tokens": 559,
"output_tokens": 399,
"input_tokens": 160,
"output_tokens_details": {
"text_tokens": 399
}
},
"request_id": "d532f72c-e75b-4ffb-a1ef-d2465e758958"
}
Cara kerja
- Interaksi satu arah: Setiap permintaan bersifat independen. Percakapan multi-arah tidak didukung.
- Tugas tetap: Hanya menghasilkan deskripsi audio dalam bahasa Inggris. Instruksi seperti pesan sistem tidak dapat mengubah perilaku, format keluaran, atau fokus konten.
-
Hanya input audio: Menerima hanya audio—tidak ada prompt teks. Format parameter
messagebersifat tetap.Contoh format message
Copymessages=[ { "role": "user", "content": [ { "type": "input_audio", "input_audio": { "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav" } } ] } ]
Keluaran streaming
Keluaran streaming mengembalikan hasil secara bertahap saat dihasilkan, sehingga mengurangi waktu tunggu.- Kompatibel dengan OpenAI
- DashScope
Atur
stream ke true untuk mengaktifkan keluaran streaming.Python
Copy
import os
from openai import OpenAI
client = OpenAI(
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-omni-30b-a3b-captioner",
messages=[
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"
}
}
]
}
],
stream=True,
stream_options={"include_usage": True},
)
for chunk in completion:
# Jika stream_options.include_usage bernilai True, bidang choices pada chunk terakhir adalah daftar kosong dan harus dilewati. Anda dapat memperoleh penggunaan token dari chunk.usage.
if chunk.choices and chunk.choices[0].delta.content != "":
print(chunk.choices[0].delta.content,end="")
Panggil melalui SDK DashScope atau HTTP. Atur parameter sesuai metode Anda:
- SDK Python: Atur parameter
streamke True. - SDK Java: Gunakan metode
streamCall. - HTTP: Di header, atur
X-DashScope-SSEkeenable.
Secara default, setiap chunk berisi seluruh konten yang telah dihasilkan sebelumnya. Untuk output inkremental, aturincremental_output(incrementalOutputdalam Java) ketrue.
Python
Copy
import dashscope
# URL berikut untuk wilayah Singapura. Jika Anda menggunakan model di wilayah Beijing, ganti URL dengan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1"
messages = [
{
"role": "user",
"content": [
{"audio": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20240916/xvappi/%E8%A3%85%E4%BF%AE%E5%99%AA%E9%9F%B3.wav"}
]
}
]
response = dashscope.MultiModalConversation.call(
# Jika variabel lingkungan belum dikonfigurasi, ganti dengan Kunci API Anda: api_key="sk-xxx"
# Kunci API berbeda berdasarkan wilayah. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv('DASHSCOPE_API_KEY'),
model="qwen3-omni-30b-a3b-captioner",
messages=messages,
stream=True,
incremental_output=True
)
full_content = ""
print("Streaming output:")
for response in response:
if response["output"]["choices"][0]["message"].content:
print(response["output"]["choices"][0]["message"].content[0]["text"])
full_content += response["output"]["choices"][0]["message"].content[0]["text"]
print(f"Full content: {full_content}")
Kirimkan file lokal (encoding Base64 atau jalur file)
Dua metode untuk mengunggah file lokal:- Gunakan encoding Base64
- Jalur file langsung (Disarankan untuk stabilitas transmisi yang lebih baik)
- Kirimkan melalui jalur file
- Penerusan melalui pengodean Base64
Kirimkan jalur file secara langsung. Didukung hanya oleh SDK Python dan Java DashScope, bukan HTTP. Format jalur berbeda berdasarkan SDK dan OS.
Tentukan jalur file
Tentukan jalur file
Sistem | SDK | Jalur file masukan | Contoh |
|---|---|---|---|
Linux atau macOS | Python SDK | file://{absolute_path_of_the_file} | file:///home/images/test.mp3 |
Java SDK | |||
Sistem operasi Windows | Python SDK | file://{absolute_path_of_the_file} | file://D:/images/test.mp3 |
Java SDK | file:///{absolute_path_of_the_file} | file:///D:/images/test.mp3 |
Konversi file menjadi string Base64 dan kirimkan ke model.
Langkah-langkah mengirimkan string Base64
Langkah-langkah mengirimkan string Base64
-
Encode file: Konversi file audio lokal menjadi string Base64.
Contoh: Mengonversi file audio menjadi string Base64
Copyimport base64 # Fungsi encoding: Mengonversi file lokal menjadi string Base64 def encode_audio(audio_path): with open(audio_path, "rb") as audio_file: return base64.b64encode(audio_file.read()).decode("utf-8") # Ganti xxxx/test.mp3 dengan jalur mutlak file audio lokal Anda base64_audio = encode_audio("xxxx/test.mp3") -
Buat Data URL:
data:;base64,{base64_audio}, di manabase64_audioadalah string Base64 dari langkah 1. -
Kirimkan Data URL melalui parameter
audio(DashScope) atauinput_audio(kompatibel OpenAI).
- Jalur file direkomendasikan untuk stabilitas transmisi. Base64 juga berfungsi untuk file di bawah 1 MB.
- Saat mengirimkan melalui jalur file, file audio harus di bawah 10 MB.
- Saat menggunakan Base64, string yang diencode harus di bawah 10 MB. Catatan: Base64 memperbesar ukuran file.
- Melewatkan berdasarkan jalur file
- Kirimkan melalui encoding Base64
Pengiriman melalui jalur file hanya didukung oleh SDK Python dan Java DashScope, bukan HTTP.
Python
Copy
import dashscope
import os
# URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'
# Ganti ABSOLUTE_PATH/welcome.mp3 dengan jalur mutlak file audio lokal Anda.
# Jalur lengkap file lokal harus diawali dengan file:// untuk memastikan jalur valid, contoh: file:///home/images/test.mp3
audio_file_path = "file://ABSOLUTE_PATH/welcome.mp3"
messages = [
{
"role": "user",
# Kirimkan jalur file yang diawali dengan file:// dalam parameter audio.
"content": [{"audio": audio_file_path}],
}
]
response = dashscope.MultiModalConversation.call(
# Kunci API berbeda berdasarkan wilayah. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# Jika Anda belum mengonfigurasi variabel lingkungan, ganti baris berikut dengan Kunci API Model Studio Anda: api_key="sk-xxx"
api_key=os.getenv('DASHSCOPE_API_KEY'),
model="qwen3-omni-30b-a3b-captioner",
messages=messages)
print("Output:")
print(response["output"]["choices"][0]["message"].content[0]["text"])
- Kompatibel dengan OpenAI
- DashScope
- Python
- Node.js
- curl
Copy
import os
from openai import OpenAI
import base64
client = OpenAI(
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
def encode_audio(audio_path):
with open(audio_path, "rb") as audio_file:
return base64.b64encode(audio_file.read()).decode("utf-8")
# Ganti ABSOLUTE_PATH/welcome.mp3 dengan jalur mutlak file audio lokal Anda.
audio_file_path = "xxx/ABSOLUTE_PATH/welcome.mp3"
base64_audio = encode_audio(audio_file_path)
completion = client.chat.completions.create(
model="qwen3-omni-30b-a3b-captioner",
messages=[
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
# Saat mengirimkan file lokal dengan encoding Base64, Anda harus menggunakan awalan data: untuk memastikan URL file valid.
# Kata kunci "base64" harus disertakan sebelum data Base64 (base64_audio), jika tidak akan terjadi error.
"data": f"data:;base64,{base64_audio}"
},
}
],
},
]
)
print(completion.choices[0].message.content)
Copy
import OpenAI from "openai";
import { readFileSync } from 'fs';
const openai = new OpenAI(
{
// Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
apiKey: process.env.DASHSCOPE_API_KEY,
// URL berikut untuk wilayah Singapura. Ganti {WorkspaceId} dengan ID ruang kerja aktual Anda. Untuk Beijing, gunakan URL yang ditampilkan di tab Tiongkok (Beijing).
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
const encodeAudio = (audioPath) => {
const audioFile = readFileSync(audioPath);
return audioFile.toString('base64');
};
// Ganti ABSOLUTE_PATH/welcome.mp3 dengan jalur mutlak file audio lokal Anda.
const base64Audio = encodeAudio("xxx/ABSOLUTE_PATH/welcome.mp3")
const completion = await openai.chat.completions.create({
model: "qwen3-omni-30b-a3b-captioner",
messages: [
{
"role": "user",
"content": [{
"type": "input_audio",
"input_audio": { "data": `data:;base64,${base64Audio}`}
}]
}]
});
console.log(completion.choices[0].message.content);
- Konversi file ke Base64 menggunakan contoh kode.
- String
"data:;base64,SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4LjI5...."dipotong untuk demonstrasi. Kirimkan string yang diencode lengkap dalam praktiknya.
Copy
# ======= Penting =======
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# base_url wilayah Singapura. Untuk Beijing, gunakan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions
# === Hapus komentar ini sebelum eksekusi ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3-omni-30b-a3b-captioner",
"messages": [
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "data:;base64,SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4LjI5...."
}
}
]
}
]
}'
- Python
- Java
- curl
Copy
import os
import base64
import dashscope
dashscope.base_http_api_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1"
# Fungsi encoding: Mengonversi file lokal menjadi string Base64
def encode_audio(audio_file_path):
with open(audio_file_path, "rb") as audio_file:
return base64.b64encode(audio_file.read()).decode("utf-8")
# Ganti ABSOLUTE_PATH/welcome.mp3 dengan jalur mutlak file audio lokal Anda.
audio_file_path = "xxx/ABSOLUTE_PATH/welcome.mp3"
base64_audio = encode_audio(audio_file_path)
print(base64_audio)
messages = [
{
"role": "user",
# Saat mengirimkan file lokal dengan encoding Base64, Anda harus menggunakan awalan data: untuk memastikan URL file valid.
# Kata kunci "base64" harus disertakan sebelum data Base64 (base64_audio), jika tidak akan terjadi error.
"content": [{"audio":f"data:;base64,{base64_audio}"}],
}
]
response = dashscope.MultiModalConversation.call(
# Jika Anda belum mengonfigurasi variabel lingkungan, ganti baris berikut dengan Kunci API Model Studio Anda: api_key="sk-xxx"
# Kunci API berbeda berdasarkan wilayah. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
model="qwen3-omni-30b-a3b-captioner",
messages=messages,
)
print(response.output.choices[0].message.content[0]["text"])
Copy
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
// Berikut adalah base_url untuk wilayah Singapura. Jika Anda menggunakan model di wilayah Beijing, ganti base_url dengan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
private static String encodeAudioToBase64(String audioPath) throws IOException {
Path path = Paths.get(audioPath);
byte[] audioBytes = Files.readAllBytes(path);
return Base64.getEncoder().encodeToString(audioBytes);
}
public static void callWithLocalFile()
throws ApiException, NoApiKeyException, UploadFileException,IOException{
// Ganti ABSOLUTE_PATH/welcome.mp3 dengan jalur aktual file lokal Anda.
String localFilePath = "ABSOLUTE_PATH/welcome.mp3";
String base64Audio = encodeAudioToBase64(localFilePath);
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
// Saat mengirimkan file lokal dengan encoding Base64, Anda harus menggunakan awalan data: untuk memastikan URL file valid.
// Kata kunci "base64" harus disertakan sebelum data Base64 (base64_audio), jika tidak akan terjadi error.
.content(Arrays.asList(
new HashMap<String, Object>(){{put("audio", "data:;base64," + base64Audio);}}
))
.build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
.model("qwen3-omni-30b-a3b-captioner")
// Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
// Jika Anda belum mengonfigurasi variabel lingkungan, ganti baris berikut dengan Kunci API Model Studio Anda: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.message(userMessage)
.build();
MultiModalConversationResult result = conv.call(param);
System.out.println("Output:\n" + result.getOutput().getChoices().get(0).getMessage().getContent().get(0).get("text"));
}
public static void main(String[] args) {
try {
callWithLocalFile();
} catch (ApiException | NoApiKeyException | UploadFileException | IOException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
- Konversi file ke Base64 menggunakan contoh kode.
- String
"data:;base64,SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4LjI5...."dipotong untuk demonstrasi. Kirimkan string yang diencode lengkap dalam praktiknya.
Copy
# ======= Penting =======
# Kunci API untuk wilayah Singapura dan Beijing berbeda. Untuk mendapatkan Kunci API, lihat https://www.alibabacloud.com/help/en/model-studio/get-api-key
# base_url wilayah Singapura. Untuk Beijing, gunakan: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation
# === Hapus komentar ini sebelum eksekusi ===
curl -X POST 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation' \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3-omni-30b-a3b-captioner",
"input":{
"messages":[
{
"role": "user",
"content": [
{"audio": "data:;base64,SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4LjI5...."}
]
}
]
}
}'
Referensi API
Untuk parameter Qwen3-Omni-Captioner, lihat Text Generation.Kode error
Jika panggilan model gagal dan mengembalikan pesan error, lihat Kode error untuk penyelesaian.FAQ
Bagaimana cara memampatkan file audio ke ukuran yang diperlukan?
Bagaimana cara memampatkan file audio ke ukuran yang diperlukan?
- Online: Gunakan Compresss untuk memampatkan audio.
- Kode: Gunakan FFmpeg. Unduh dari situs web resmi FFmpeg.
Copy
# Perintah konversi dasar (templat universal)
# -i: Menentukan jalur file input. Contoh: input.mp3
# -b:a: Mengatur bitrate audio.
# Nilai umum: 64 kbps (kualitas rendah, untuk suara dan streaming bandwidth rendah), 128k (kualitas sedang, untuk audio umum dan podcast), 192 kbps (kualitas tinggi, untuk musik dan penyiaran).
# Bitrate yang lebih tinggi menghasilkan kualitas audio lebih baik dan ukuran file lebih besar.
# -ar: Mengatur laju sampel audio, yaitu jumlah sampel per detik.
# Nilai umum: 8000 Hz, 22050 Hz, 44100 Hz (laju sampel standar).
# Laju sampel yang lebih tinggi menghasilkan ukuran file lebih besar.
# -ac: Mengatur jumlah saluran audio. Nilai umum: 1 (mono), 2 (stereo). File mono lebih kecil.
# -y: Menimpa file output jika sudah ada (tidak perlu nilai). # output.mp3: Menentukan jalur file output.
ffmpeg -i input.mp3 -b:a 128k -ar 44100 -ac 1 output.mp3 -y
Batasan
Batasan file audio:- Durasi: Maksimal 40 menit.
- Jumlah file: Hanya satu file audio yang didukung per permintaan.
- Format file: AMR, WAV (CodecID: GSM_MS), WAV (PCM), 3GP, 3GPP, AAC, dan MP3.
- Metode input file: URL publik, encoding Base64, atau jalur file lokal.
-
Ukuran file:
- URL publik: Tidak lebih dari 1 GB.
- Jalur file: File audio harus lebih kecil dari 10 MB.
- Encoding Base64: String yang diencode harus di bawah 10 MB. Kirimkan file lokal.
Untuk memampatkan file, lihat Bagaimana cara memampatkan file audio ke ukuran yang diperlukan?