Dans les applications de chat en temps réel ou de génération de texte long, des temps d'attente prolongés dégradent l'expérience utilisateur et peuvent provoquer des délais d'expiration côté serveur, entraînant l'échec des tâches. Le streaming output résout ces problèmes en renvoyant continûment des fragments de texte au fur et à mesure que le modèle les génère.
Fonctionnement
Le streaming output utilise le protocole Server-Sent Events (SSE). Lorsqu'une requête en streaming démarre, le serveur établit une connexion HTTP persistante avec le client. Chaque fois que le modèle génère un bloc de texte (appelé chunk), il le transmet immédiatement via cette connexion. Une fois tout le contenu généré, le serveur envoie un signal de fin. Le client écoute le flux d'événements et reçoit les chunks de texte en temps réel pour les traiter, par exemple en affichant les caractères un par un dans l'interface. Cette approche diffère des appels non streamés, qui renvoient l'intégralité du contenu en une seule fois.Facturation
Le streaming output suit les mêmes règles de facturation que les appels non streamés, avec une tarification basée sur le nombre de tokens d'entrée et de tokens de sortie dans la requête. Si une requête est interrompue, seuls les tokens de sortie générés avant que le serveur ne reçoive la demande d'arrêt sont comptabilisés.Utilisation
La version open source de Qwen3, les versions commerciale et open source de QwQ, ainsi que QVQ et Qwen-Omni, prennent uniquement en charge le streaming output.
Étape 1 : Configurez votre clé API et sélectionnez une région
Vous devez avoir obtenu une clé API et l'avoir configurée en tant que variable d'environnement.
La configuration de votre clé API en tant que variable d'environnement ( DASHSCOPE_API_KEY ) est plus sécurisée que son codage en dur dans votre code.
Étape 2 : Effectuez une requête en streaming
- OpenAI compatible
- DashScope
-
Activation
Définissez
streamsurtrue. -
Affichage de l'utilisation des tokens
Le protocole OpenAI ne renvoie pas l'utilisation des tokens par défaut. Définissez
stream_options={"include_usage": true}afin que le dernier chunk de données renvoyé inclue les informations d'utilisation des tokens.
- Python
- Node.js
- curl
Copy
import os
from openai import OpenAI
# 1. Prepare: Initialize the client
client = OpenAI(
# Configure the API key using an environment variable to avoid hard coding.
api_key=os.environ["DASHSCOPE_API_KEY"],
# The API key is tightly bound to a region. Ensure base_url matches the region of your API key.
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
# 2. Make a streaming request
completion = client.chat.completions.create(
model="qwen-plus",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Please introduce yourself"}
],
stream=True,
stream_options={"include_usage": True}
)
# 3. Handle the streaming response
# Store response fragments in a list. Joining them at the end is more efficient than repeated string concatenation.
content_parts = []
print("AI: ", end="", flush=True)
for chunk in completion:
if chunk.choices:
content = chunk.choices[0].delta.content or ""
print(content, end="", flush=True)
content_parts.append(content)
elif chunk.usage:
print("\n--- Request usage ---")
print(f"Input Tokens: {chunk.usage.prompt_tokens}")
print(f"Output Tokens: {chunk.usage.completion_tokens}")
print(f"Total Tokens: {chunk.usage.total_tokens}")
full_response = "".join(content_parts)
# print(f"\n--- Full response ---\n{full_response}")
Réponse
Copy
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 87
Total Tokens: 113
Copy
import OpenAI from "openai";
async function main() {
// 1. Prepare: Initialize the client
// Configure the API key using an environment variable to avoid hard coding.
if (!process.env.DASHSCOPE_API_KEY) {
throw new Error("Set the DASHSCOPE_API_KEY environment variable");
}
// The API key is tightly bound to a region. Ensure baseURL matches the region of your API key.
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
const client = new OpenAI({
apiKey: process.env.DASHSCOPE_API_KEY,
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
});
try {
// 2. Make a streaming request
const stream = await client.chat.completions.create({
model: "qwen-plus",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "Please introduce yourself" },
],
stream: true,
// Purpose: Get token usage in the last chunk.
stream_options: { include_usage: true },
});
// 3. Handle the streaming response
const contentParts = [];
process.stdout.write("AI: ");
for await (const chunk of stream) {
// The last chunk contains no choices but includes usage information.
if (chunk.choices && chunk.choices.length > 0) {
const content = chunk.choices[0]?.delta?.content || "";
process.stdout.write(content);
contentParts.push(content);
} else if (chunk.usage) {
// Request complete. Print token usage.
console.log("\n--- Request usage ---");
console.log(`Input Tokens: ${chunk.usage.prompt_tokens}`);
console.log(`Output Tokens: ${chunk.usage.completion_tokens}`);
console.log(`Total Tokens: ${chunk.usage.total_tokens}`);
}
}
const fullResponse = contentParts.join("");
// console.log(`\n--- Full response ---\n${fullResponse}`);
} catch (error) {
console.error("Request failed:", error);
}
}
main();
Réponse
Copy
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 89
Total Tokens: 115
Requête
Copy
# ======= Important notes =======
# Ensure the DASHSCOPE_API_KEY environment variable is set
# API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
--no-buffer \
-d '{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "Who are you?"}
],
"stream": true,
"stream_options": {"include_usage": true}
}'
Réponse
La réponse suit le protocole SSE. Chaque ligne commençant pardata: représente un fragment de données.Copy
data: {"choices":[{"delta":{"content":"","role":"assistant"},"index":0,"logprobs":null,"finish_reason":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"finish_reason":null,"delta":{"content":"I am"},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" from"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" Alibaba"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":"'s large-scale language"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" model, my name is Qwen"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":"."},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"finish_reason":"stop","delta":{"content":""},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[],"object":"chat.completion.chunk","usage":{"prompt_tokens":22,"completion_tokens":17,"total_tokens":39},"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: [DONE]
data:: la charge utile du message, généralement une chaîne JSON.[DONE]: indique la fin de l'ensemble de la réponse en streaming.
-
Comment activer
La méthode d'activation de la sortie en streaming varie selon le SDK ou l'outil utilisé :
- SDK Python : définissez le paramètre
streamsurTrue. - SDK Java : utilisez l'interface
streamCall. - curl : définissez l'en-tête
X-DashScope-SSEsurenable.
- SDK Python : définissez le paramètre
-
Activer la sortie incrémentielle
Le protocole DashScope prend en charge les sorties en streaming incrémentielles et non incrémentielles :
-
Incrémentiel (recommandé) : chaque fragment de données contient uniquement le contenu nouvellement généré. Définissez
incremental_outputsurtruepour activer la sortie en streaming incrémentielle (le SDK Python 2.0 génère par défaut une sortie incrémentielle ; ce paramètre est inutile).Exemple : ["J'aime","manger","des pommes"]
-
Non incrémentiel : chaque fragment de données contient tout le contenu précédemment généré, ce qui gaspille la bande passante réseau et augmente la charge de traitement côté client. Définissez
incremental_outputsurfalsepour activer la sortie en streaming non incrémentielle.Exemple : ["J'aime","J'aime manger","J'aime manger des pommes"]
-
Incrémentiel (recommandé) : chaque fragment de données contient uniquement le contenu nouvellement généré. Définissez
- Consulter l'utilisation des tokens Chaque fragment de données inclut des informations en temps réel sur l'utilisation des tokens.
- Python
- Java
- curl
Copy
import os
import dashscope
from dashscope.v2 import DashScopeError
# 1. Préparation : configurez l'API key et le client
# Il est recommandé de configurer l'API key via une variable d'environnement pour éviter de la coder en dur.
# Voici la configuration de la région de Singapour (ap-southeast-1). Remplacez {WorkspaceId} par l'ID de votre workspace. La valeur de region varie selon la région.
try:
api_key = os.environ["DASHSCOPE_API_KEY"]
except KeyError:
raise ValueError("Définissez la variable d'environnement DASHSCOPE_API_KEY")
client = dashscope.Client(api_key=api_key, workspace="{WorkspaceId}", region="ap-southeast-1")
# 2. Envoi d'une requête en streaming
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Please introduce yourself"},
]
try:
stream = client.chat.completions.create(
model="qwen-plus",
messages=messages,
stream=True,
# La sortie en streaming est incrémentale par défaut : chaque chunk ne contient que le contenu nouvellement généré.
)
# 3. Traitement de la réponse en streaming
content_parts = []
print("AI: ", end="", flush=True)
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content, end="", flush=True)
content_parts.append(delta.content)
# Vérifie s'il s'agit du dernier chunk (le dernier chunk contient l'utilisation des tokens)
if chunk.choices[0].finish_reason == "stop":
usage = chunk.usage
print("\n--- Utilisation de la requête ---")
print(f"Tokens d'entrée: {usage.input_tokens}")
print(f"Tokens de sortie: {usage.output_tokens}")
print(f"Tokens au total: {usage.total_tokens}")
full_response = "".join(content_parts)
# print(f"\n--- Réponse complète ---\n{full_response}")
except DashScopeError as e:
# Gestion des erreurs (les erreurs survenues pendant le streaming sont directement levées sous forme d'exceptions)
print(
f"\nÉchec de la requête: request_id={e.request_id}, code={e.code}, message={e.message}"
)
Copy
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can help you answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 91
Total Tokens: 117
Copy
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import io.reactivex.Flowable;
import io.reactivex.schedulers.Schedulers;
import java.util.Arrays;
import java.util.concurrent.CountDownLatch;
import com.alibaba.dashscope.protocol.Protocol;
public class Main {
public static void main(String[] args) {
// 1. Get the API key
String apiKey = System.getenv("DASHSCOPE_API_KEY");
if (apiKey == null || apiKey.isEmpty()) {
System.err.println("Set the DASHSCOPE_API_KEY environment variable");
return;
}
// 2. Initialize the Generation instance
// The API key is tightly bound to a region. Ensure baseUrl matches the region of your API key.
Generation gen = new Generation(Protocol.HTTP.getValue(), "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1");
CountDownLatch latch = new CountDownLatch(1);
// 3. Build request parameters
GenerationParam param = GenerationParam.builder()
.apiKey(apiKey)
.model("qwen-plus")
.messages(Arrays.asList(
Message.builder()
.role(Role.USER.getValue())
.content("Introduce yourself")
.build()
))
.resultFormat(GenerationParam.ResultFormat.MESSAGE)
.incrementalOutput(true) // Enable incremental output for streaming
.build();
// 4. Make a streaming call and handle the response
try {
Flowable<GenerationResult> result = gen.streamCall(param);
StringBuilder fullContent = new StringBuilder();
System.out.print("AI: ");
result
.subscribeOn(Schedulers.io()) // Execute request on IO thread
.observeOn(Schedulers.computation()) // Process response on computation thread
.subscribe(
// onNext: Handle each response fragment
message -> {
String content = message.getOutput().getChoices().get(0).getMessage().getContent();
String finishReason = message.getOutput().getChoices().get(0).getFinishReason();
// Output content
System.out.print(content);
fullContent.append(content);
// When finishReason is not null, it indicates the last chunk. Output usage info.
if (finishReason != null && !"null".equals(finishReason)) {
System.out.println("\n--- Request usage ---");
System.out.println("Input Tokens: " + message.getUsage().getInputTokens());
System.out.println("Output Tokens: " + message.getUsage().getOutputTokens());
System.out.println("Total Tokens: " + message.getUsage().getTotalTokens());
}
System.out.flush(); // Flush output immediately
},
// onError: Handle errors
error -> {
System.err.println("\nRequest failed: " + error.getMessage());
latch.countDown();
},
// onComplete: Completion callback
() -> {
System.out.println(); // New line
// System.out.println("Full response: " + fullContent.toString());
latch.countDown();
}
);
// Main thread waits for the async task to complete
latch.await();
System.out.println("Program execution complete");
} catch (Exception e) {
System.err.println("Request exception: " + e.getMessage());
e.printStackTrace();
}
}
}
Réponse
Copy
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can help you answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 91
Total Tokens: 117
Requête
Copy
# ======= Important notes =======
# Ensure the DASHSCOPE_API_KEY environment variable is set
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-H "X-DashScope-SSE: enable" \
-d '{
"model": "qwen-plus",
"input":{
"messages":[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Who are you?"
}
]
},
"parameters": {
"result_format": "message",
"incremental_output":true
}
}'
Réponse
La réponse suit le format Server-Sent Events (SSE). Chaque message comprend :- id : numéro du fragment de données.
- event : type d'événement, toujours « result ».
- Informations sur le code d'état HTTP.
- data : données au format JSON.
Copy
id:1
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"I am","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":27,"output_tokens":1,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:2
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"Qwen","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":30,"output_tokens":4,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:3
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":" from Alibaba","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":33,"output_tokens":7,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
...
id:13
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"or need help, feel free to","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":90,"output_tokens":64,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:14
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"ask me!","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":92,"output_tokens":66,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:15
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","role":"assistant"},"finish_reason":"stop"}]},"usage":{"total_tokens":92,"output_tokens":66,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
Sortie en streaming pour les modèles multimodaux
Les modèles multimodaux permettent d'ajouter des images, de l'audio et d'autres contenus aux conversations. Leur implémentation de la sortie en streaming diffère de celle des modèles textuels purs sur les points suivants :- Construction du message utilisateur : les entrées des modèles multimodaux incluent non seulement du texte, mais aussi des images, de l'audio et d'autres informations multimodales.
- Interface du SDK DashScope : utilisez l'interface MultiModalConversation dans le SDK Python DashScope. Utilisez la classe MultiModalConversation dans le SDK Java DashScope.
Pour les modèles multimodaux, consultez Compréhension d'images et de vidéos , Extraction de texte , Compréhension audio — Qwen3-Omni-Captioner , Kimi , etc. Le modèle Qwen-Omni prend uniquement en charge la sortie en streaming car sa sortie peut inclure du texte ou de l' audio et d'autres contenus multimodaux. L'analyse de ses résultats diffère de celle des autres modèles. Pour plus de détails, consultez Omni-modal .
- OpenAI compatible
- DashScope
- Python
- Node.js
- curl
Copy
from openai import OpenAI
import os
client = OpenAI(
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx"
# API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-vl-plus", # Replace with other multimodal models as needed and adjust messages accordingly
messages=[
{"role": "user",
"content": [{"type": "image_url",
"image_url": {"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream=True,
# stream_options={"include_usage": True}
)
full_content = ""
print("Streaming output content:")
for chunk in completion:
# If stream_options.include_usage is True, the last chunk's choices field is an empty list and should be skipped (token usage can be obtained via chunk.usage)
if chunk.choices and chunk.choices[0].delta.content != "":
full_content += chunk.choices[0].delta.content
print(chunk.choices[0].delta.content)
print(f"Full content: {full_content}")
Copy
from openai import OpenAI
import os
client = OpenAI(
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx"
api_key=os.getenv("DASHSCOPE_API_KEY"),
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-vl-plus", # Replace with other multimodal models as needed and adjust messages accordingly
messages=[
{"role": "user",
"content": [{"type": "image_url",
"image_url": {"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream=True,
# stream_options={"include_usage": True}
)
full_content = ""
print("Streaming output content:")
for chunk in completion:
# If stream_options.include_usage is True, the last chunk's choices field is an empty list and should be skipped (token usage can be obtained via chunk.usage)
if chunk.choices and chunk.choices[0].delta.content != "":
full_content += chunk.choices[0].delta.content
print(chunk.choices[0].delta.content)
print(f"Full content: {full_content}")
Copy
import OpenAI from "openai";
const openai = new OpenAI(
{
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: apiKey: "sk-xxx"
// API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
apiKey: process.env.DASHSCOPE_API_KEY,
// China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
baseURL: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"
}
);
const completion = await openai.chat.completions.create({
model: "qwen3-vl-plus", // Replace with other multimodal models as needed and adjust messages accordingly
messages: [
{role: "user",
content: [{"type": "image_url",
"image_url": {"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream: true,
// stream_options: { include_usage: true },
});
let fullContent = ""
console.log("Streaming output content:")
for await (const chunk of completion) {
// If stream_options.include_usage is true, the last chunk's choices field is an empty array and should be skipped (token usage can be obtained via chunk.usage)
if (chunk.choices[0] && chunk.choices[0].delta.content != null) {
fullContent += chunk.choices[0].delta.content;
console.log(chunk.choices[0].delta.content);
}
}
console.log(`Full output content: ${fullContent}`)
Copy
import OpenAI from "openai";
const openai = new OpenAI(
{
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: apiKey: "sk-xxx"
apiKey: process.env.DASHSCOPE_API_KEY,
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
const completion = await openai.chat.completions.create({
model: "qwen3-vl-plus", // Replace with other multimodal models as needed and adjust messages accordingly
messages: [
{role: "user",
content: [{"type": "image_url",
"image_url": {"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream: true,
// stream_options: { include_usage: true },
});
let fullContent = ""
console.log("Streaming output content:")
for await (const chunk of completion) {
// If stream_options.include_usage is true, the last chunk's choices field is an empty array and should be skipped (token usage can be obtained via chunk.usage)
if (chunk.choices[0] && chunk.choices[0].delta.content != null) {
fullContent += chunk.choices[0].delta.content;
console.log(chunk.choices[0].delta.content);
}
}
console.log(`Full output content: ${fullContent}`)
Copy
# ======= Important notes =======
# API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "qwen3-vl-plus",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"
}
},
{
"type": "text",
"text": "What scene is depicted in the image?"
}
]
}
],
"stream":true,
"stream_options":{"include_usage":true}
}'
Copy
# ======= Important notes =======
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# === Delete this comment before running ===
curl --location 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "qwen3-vl-plus",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"
}
},
{
"type": "text",
"text": "What scene is depicted in the image?"
}
]
}
],
"stream":true,
"stream_options":{"include_usage":true}
}'
- Python
- Java
- curl
Copy
import os
import dashscope
# Voici la configuration de la région de la Chine (Pékin). Lors de l'appel, remplacez {WorkspaceId} par l'ID de votre workspace. La valeur de region varie selon la région.
client = dashscope.Client(
# Si la variable d'environnement n'est pas configurée, remplacez la ligne suivante par votre API Key de Model Studio : api_key = "sk-xxx"
# Les API Key sont spécifiques à chaque région. Pour obtenir une API Key, consultez https://www.alibabacloud.com/help/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
workspace="{WorkspaceId}",
region="cn-beijing",
)
messages = [
{
"role": "user",
"content": [
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
stream = client.multimodal.chat.create(
model='qwen3-vl-plus', # Vous pouvez utiliser un autre modèle multimodal selon vos besoins et adapter les messages en conséquence
messages=messages,
stream=True,
)
full_content = ""
print("Contenu de la sortie en streaming:")
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content)
full_content += delta.content
print(f"Contenu complet: {full_content}")
Copy
import os
import dashscope
# Voici la configuration de la région de Singapour (ap-southeast-1). Remplacez {WorkspaceId} par l'ID de votre workspace. La valeur de region varie selon la région.
client = dashscope.Client(
# Si la variable d'environnement n'est pas configurée, remplacez la ligne suivante par votre API Key de Model Studio : api_key = "sk-xxx"
# Les API Key sont spécifiques à chaque région. Pour obtenir une API Key, consultez https://www.alibabacloud.com/help/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
workspace="{WorkspaceId}",
region="ap-southeast-1",
)
messages = [
{
"role": "user",
"content": [
{"image": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
stream = client.multimodal.chat.create(
model='qwen3-vl-plus', # Vous pouvez utiliser un autre modèle multimodal selon vos besoins et adapter les messages en conséquence
messages=messages,
stream=True,
)
full_content = ""
print("Contenu de la sortie en streaming:")
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content)
full_content += delta.content
print(f"Contenu complet: {full_content}")
Copy
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import io.reactivex.Flowable;
import com.alibaba.dashscope.utils.Constants;
public class Main {
// China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
static {Constants.baseHttpApiUrl="https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1";}
public static void streamCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("image", "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"),
Collections.singletonMap("text", "What scene is depicted in the image?"))).build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: .apiKey("sk-xxx")
// API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3-vl-plus") // Replace with other multimodal models as needed and adjust messages accordingly
.messages(Arrays.asList(userMessage))
.incrementalOutput(true)
.build();
Flowable<MultiModalConversationResult> result = conv.streamCall(param);
result.blockingForEach(item -> {
try {
List<Map<String, Object>> content = item.getOutput().getChoices().get(0).getMessage().getContent();
// Check if content exists and is not empty
if (content != null && !content.isEmpty()) {
System.out.println(content.get(0).get("text"));
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
});
}
public static void main(String[] args) {
try {
streamCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
Copy
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import io.reactivex.Flowable;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
public static void streamCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
// must create mutable map.
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("image", "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"),
Collections.singletonMap("text", "What scene is depicted in the image?"))).build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3-vl-plus") // Replace with other multimodal models as needed and adjust messages accordingly
.messages(Arrays.asList(userMessage))
.incrementalOutput(true)
.build();
Flowable<MultiModalConversationResult> result = conv.streamCall(param);
result.blockingForEach(item -> {
try {
List<Map<String, Object>> content = item.getOutput().getChoices().get(0).getMessage().getContent();
// Check if content exists and is not empty
if (content != null && !content.isEmpty()) {
System.out.println(content.get(0).get("text"));
}
} catch (Exception e){
System.exit(0);
}
});
}
public static void main(String[] args) {
try {
streamCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
Copy
# ======= Important notes =======
# API keys differ by region. Get your API key: https://help.aliyun.com/zh/model-studio/get-api-key
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-H 'X-DashScope-SSE: enable' \
-d '{
"model": "qwen3-vl-plus",
"input":{
"messages":[
{
"role": "user",
"content": [
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
},
"parameters": {
"incremental_output": true
}
}'
Copy
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-H 'X-DashScope-SSE: enable' \
-d '{
"model": "qwen3-vl-plus",
"input":{
"messages":[
{
"role": "user",
"content": [
{"image": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
},
"parameters": {
"incremental_output": true
}
}'
Sortie en continu pour les modèles de raisonnement
Les modèles de raisonnement renvoient d'abord le champreasoning_content (le processus de réflexion), puis le champ content (la réponse). Déterminez si l'étape actuelle correspond à la phase de réflexion ou de réponse en fonction du statut du paquet de données.
Pour plus d'informations sur les modèles de raisonnement, consultez les rubriques Réflexion approfondie et Compréhension des images et des vidéos.
Pour la mise en œuvre de la sortie en continu de Qwen3-Omni-Flash (mode réflexion), reportez-vous à la section Omni-modal .
- OpenAI compatible
- DashScope
Voici le format de réponse lors de l'appel au mode de réflexion du modèle qwen-plus à l'aide du SDK Python OpenAI en mode streaming :
Copy
# Thinking stage
...
ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content='Cover all key points while')
ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content='remaining natural and fluent.')
# Response stage
ChoiceDelta(content='Hello! I am **Qwen', function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content=None)
ChoiceDelta(content='** (', function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content=None)
...
- Si
reasoning_contentn'est pas None et quecontentestNone, l'étape actuelle correspond à la phase de réflexion. - Si
reasoning_contentest None et quecontentn'est pasNone, l'étape actuelle correspond à la phase de réponse. - Si les deux champs sont
None, l'étape reste identique à celle du paquet précédent.
- Python
- Node.js
- HTTP
Exemple de code
Copy
from openai import OpenAI
import os
# Initialize the OpenAI client
client = OpenAI(
# If you haven't configured an environment variable, replace with your Alibaba Cloud Model Studio API key: api_key="sk-xxx"
api_key=os.getenv("DASHSCOPE_API_KEY"),
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
messages = [{"role": "user", "content": "Who are you"}]
completion = client.chat.completions.create(
model="qwen-plus", # Replace with other deep-thinking models as needed
messages=messages,
# The enable_thinking parameter enables the thinking process. This parameter has no effect on models QwQ.
extra_body={"enable_thinking": True},
stream=True,
# stream_options={
# "include_usage": True
# },
)
reasoning_content = "" # Full thought process
answer_content = "" # Full response
is_answering = False # Whether in the response stage
print("\n" + "=" * 20 + "Thought process" + "=" * 20 + "\n")
for chunk in completion:
if not chunk.choices:
print("\nUsage:")
print(chunk.usage)
continue
delta = chunk.choices[0].delta
# Collect only thinking content
if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None:
if not is_answering:
print(delta.reasoning_content, end="", flush=True)
reasoning_content += delta.reasoning_content
# Received content, start responding
if hasattr(delta, "content") and delta.content:
if not is_answering:
print("\n" + "=" * 20 + "Full response" + "=" * 20 + "\n")
is_answering = True
print(delta.content, end="", flush=True)
answer_content += delta.content
Réponse
Copy
====================Thought process====================
Okay, the user asked "Who are you," so I need to give an accurate and friendly answer. First, I should confirm my identity as Qwen, developed by Tongyi Lab under Alibaba Group. Next, explain my main functions, like answering questions, creating text, logical reasoning, etc. Keep the tone approachable and avoid overly technical terms so the user feels comfortable. Also, avoid complex jargon and ensure the answer is concise. Additionally, include some interactive elements to encourage further questions. Finally, check for any missing key information, such as my Chinese name "Tongyi Qianwen" and English name "Qwen," along with my company and lab. Make sure the response is comprehensive and meets user expectations.
====================Full response====================
Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create text, perform logical reasoning, programming, and more, aiming to provide high-quality information and services. You can call me Qwen or simply Tongyi Qianwen. How can I help you?
Exemple de code
Copy
import OpenAI from "openai";
import process from 'process';
// Initialize the openai client
const openai = new OpenAI({
apiKey: process.env.DASHSCOPE_API_KEY, // Read from environment variable
baseURL: 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1'
});
let reasoningContent = '';
let answerContent = '';
let isAnswering = false;
async function main() {
try {
const messages = [{ role: 'user', content: 'Who are you' }];
const stream = await openai.chat.completions.create({
// Replace with other Qwen3 models or QwQ models as needed
model: 'qwen-plus',
messages,
stream: true,
// The enable_thinking parameter enables the thinking process. This parameter has no effect on models QwQ.
enable_thinking: true
});
console.log('\n' + '='.repeat(20) + 'Thought process' + '='.repeat(20) + '\n');
for await (const chunk of stream) {
if (!chunk.choices?.length) {
console.log('\nUsage:');
console.log(chunk.usage);
continue;
}
const delta = chunk.choices[0].delta;
// Collect only thinking content
if (delta.reasoning_content !== undefined && delta.reasoning_content !== null) {
if (!isAnswering) {
process.stdout.write(delta.reasoning_content);
}
reasoningContent += delta.reasoning_content;
}
// Received content, start responding
if (delta.content !== undefined && delta.content) {
if (!isAnswering) {
console.log('\n' + '='.repeat(20) + 'Full response' + '='.repeat(20) + '\n');
isAnswering = true;
}
process.stdout.write(delta.content);
answerContent += delta.content;
}
}
} catch (error) {
console.error('Error:', error);
}
}
main();
Réponse
Copy
====================Thought process====================
Okay, the user asked "Who are you," so I need to state my identity. First, I should clearly say I am Qwen, a large-scale language model developed by Alibaba Cloud. Next, mention my main functions, like answering questions, creating text, logical reasoning, etc. Also emphasize my multilingual support, including Chinese and English, so users know I can handle requests in different languages. Additionally, explain my application scenarios, such as helping with learning, work, and daily life. However, since the user's question is direct, detailed information might not be necessary—keep it concise. Also, ensure a friendly tone and invite further questions. Check for any missing key information, like my version or latest updates, but the user probably doesn't need that much detail. Finally, confirm the response is accurate and error-free.
====================Full response====================
I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can handle various tasks like answering questions, creating text, logical reasoning, and programming, supporting multiple languages including Chinese and English. If you have any questions or need help, feel free to tell me anytime!
Exemple de code
- curl
Pour les modèles open source Qwen3, définissez
enable_thinking sur true pour activer le mode de réflexion. Le paramètre enable_thinking n'a aucun effet sur les modèles QwQ .Copy
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-plus",
"messages": [
{
"role": "user",
"content": "Who are you"
}
],
"stream": true,
"stream_options": {
"include_usage": true
},
"enable_thinking": true
}'
Réponse
Copy
data: {"choices":[{"delta":{"content":null,"role":"assistant","reasoning_content":""},"index":0,"logprobs":null,"finish_reason":null}],"object":"chat.completion.chunk","usage":null,"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
.....
data: {"choices":[{"finish_reason":"stop","delta":{"content":"","reasoning_content":null},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
data: {"choices":[],"object":"chat.completion.chunk","usage":{"prompt_tokens":10,"completion_tokens":360,"total_tokens":370},"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
data: [DONE]
Voici le format de réponse en streaming lors de l'appel au mode de réflexion du modèle qwen-plus à l'aide du SDK Python DashScope :
Copy
# Thinking stage
...
{"role": "assistant", "content": "", "reasoning_content": "High information density,"}
{"role": "assistant", "content": "", "reasoning_content": "making users feel helped."}
# Response stage
{"role": "assistant", "content": "I am Qwen", "reasoning_content": ""}
{"role": "assistant", "content": ", developed by Tongyi Lab", "reasoning_content": ""}
...
- Si
reasoning_contentn'est pas vide et quecontentest vide, la phase actuelle est la réflexion. - Si
reasoning_contentest vide et quecontentn'est pas vide, la phase actuelle est la réponse. - Si les deux sont vides, la phase reste identique à celle du paquet précédent.
- Python
- Java
- HTTP
Exemple de code
Copy
import os
import dashscope
# Voici la configuration de la région de Singapour (ap-southeast-1). Remplacez {WorkspaceId} par l'ID de votre workspace. La valeur de region varie selon la région.
client = dashscope.Client(
# Si la variable d'environnement n'est pas configurée, remplacez la ligne suivante par votre API Key de Model Studio : api_key = "sk-xxx"
# Les API Key sont spécifiques à chaque région. Pour obtenir une API Key, consultez https://www.alibabacloud.com/help/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
workspace="{WorkspaceId}",
region="ap-southeast-1",
)
messages = [{"role": "user", "content": "Who are you?"}]
stream = client.chat.completions.create(
# Vous pouvez le remplacer par un autre modèle de réflexion profonde selon vos besoins
model="qwen-plus",
messages=messages,
# Active la réflexion profonde ; ce paramètre est sans effet sur les modèles QwQ et DeepSeek-R1
enable_thinking=True,
stream=True, # La sortie en streaming est incrémentale par défaut (les modèles open source Qwen3 ne prennent en charge que la sortie incrémentale ; elle est également recommandée pour les autres modèles pour une meilleure expérience)
)
# Définit le processus de réflexion complet
reasoning_content = ""
# Définit la réponse complète
answer_content = ""
# Détermine si le processus de réflexion est terminé et si la réponse a commencé
is_answering = False
print("=" * 20 + "Processus de réflexion" + "=" * 20)
for chunk in stream:
# Si le processus de réflexion et la réponse sont tous les deux vides, ignorer
if (
chunk.choices[0].delta.content == ""
and chunk.choices[0].delta.reasoning_content == ""
):
pass
else:
# S'il s'agit actuellement du processus de réflexion
if (
chunk.choices[0].delta.reasoning_content != ""
and chunk.choices[0].delta.content == ""
):
print(chunk.choices[0].delta.reasoning_content, end="", flush=True)
reasoning_content += chunk.choices[0].delta.reasoning_content
# S'il s'agit actuellement de la réponse
elif chunk.choices[0].delta.content != "":
if not is_answering:
print("\n" + "=" * 20 + "Réponse complète" + "=" * 20)
is_answering = True
print(chunk.choices[0].delta.content, end="", flush=True)
answer_content += chunk.choices[0].delta.content
# Pour imprimer le processus de réflexion complet et la réponse complète, décommentez les lignes suivantes et exécutez-les
# print("=" * 20 + "Processus de réflexion complet" + "=" * 20 + "\n")
# print(f"{reasoning_content}")
# print("=" * 20 + "Réponse complète" + "=" * 20 + "\n")
# print(f"{answer_content}")
Réponse
Copy
====================Thought process====================
Okay, the user asked: "Who are you?" I need to answer this question. First, clarify my identity as Qwen, a large-scale language model developed by Alibaba Cloud. Next, explain my functions and purposes, like answering questions, creating text, logical reasoning, etc. Also, emphasize my goal of being a helpful assistant.
Keep the expression conversational, avoiding professional jargon or complex sentence structures. Add friendly phrases like "Hello there~" to make the conversation natural. Also, ensure accuracy and don't omit key points like my developer, main functions, and usage scenarios.
Consider possible follow-up questions from the user, such as specific application examples or technical details, so subtly hint at further inquiries in the response. For example, mention "Whether it's everyday questions or professional issues, I'll do my best to help," which is both comprehensive and open-ended.
Finally, check for fluency, repetition, or redundant information to keep the response concise. Maintain a balance between friendliness and professionalism so users feel both approachable and reliable.
====================Full response====================
Hello there~ I'm Qwen, a large-scale language model developed by Alibaba Cloud. I can answer questions, create text, perform logical reasoning, programming, and more, aiming to provide help and support. Whether it's everyday questions or professional issues, I'll do my best to help. How can I assist you?
Exemple de code
Copy
// dashscope SDK version >= 2.19.4
import java.util.Arrays;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import io.reactivex.Flowable;
import java.lang.System;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
private static final Logger logger = LoggerFactory.getLogger(Main.class);
private static StringBuilder reasoningContent = new StringBuilder();
private static StringBuilder finalContent = new StringBuilder();
private static boolean isFirstPrint = true;
private static void handleGenerationResult(GenerationResult message) {
String reasoning = message.getOutput().getChoices().get(0).getMessage().getReasoningContent();
String content = message.getOutput().getChoices().get(0).getMessage().getContent();
if (!reasoning.isEmpty()) {
reasoningContent.append(reasoning);
if (isFirstPrint) {
System.out.println("====================Thought process====================");
isFirstPrint = false;
}
System.out.print(reasoning);
}
if (!content.isEmpty()) {
finalContent.append(content);
if (!isFirstPrint) {
System.out.println("\n====================Full response====================");
isFirstPrint = true;
}
System.out.print(content);
}
}
private static GenerationParam buildGenerationParam(Message userMsg) {
return GenerationParam.builder()
// If you haven't configured an environment variable, replace the next line with your Alibaba Cloud Model Studio API key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen-plus")
.enableThinking(true)
.incrementalOutput(true)
.resultFormat("message")
.messages(Arrays.asList(userMsg))
.build();
}
public static void streamCallWithMessage(Generation gen, Message userMsg)
throws NoApiKeyException, ApiException, InputRequiredException {
GenerationParam param = buildGenerationParam(userMsg);
Flowable<GenerationResult> result = gen.streamCall(param);
result.blockingForEach(message -> handleGenerationResult(message));
}
public static void main(String[] args) {
try {
Generation gen = new Generation();
Message userMsg = Message.builder().role(Role.USER.getValue()).content("Who are you?").build();
streamCallWithMessage(gen, userMsg);
// Print final result
// if (reasoningContent.length() > 0) {
// System.out.println("\n====================Full response====================");
// System.out.println(finalContent.toString());
// }
} catch (ApiException | NoApiKeyException | InputRequiredException e) {
logger.error("An exception occurred: {}", e.getMessage());
}
System.exit(0);
}
}
Réponse
Copy
====================Thought process====================
Okay, the user asked "Who are you?", so I need to answer based on previous settings. First, my role is Qwen, a large-scale language model under Alibaba Group. Keep it conversational and simple.
The user might be new to me or confirming my identity. Start by directly stating who I am, then briefly explain my functions and purposes, like answering questions, creating text, programming, etc. Also mention multilingual support so users know I handle different languages.
Also, per guidelines, maintain human-like qualities, so use a friendly tone, maybe add emojis for warmth. Guide users to ask further questions or use my features, like asking what they need help with.
Avoid complex terms and keep it concise. Check for missing key points like multilingual support and specific capabilities. Ensure the response meets all requirements, including conversational style and simplicity.
====================Full response====================
Hello! I'm Qwen, a large-scale language model under Alibaba Group. I can answer questions, create text like stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I'm proficient in multiple languages, including but not limited to Chinese, English, German, French, and Spanish. How can I help you?
Exemple de code
- curl
Pour les modèles de réflexion hybride, définissez
enable_thinking sur true pour activer le mode de réflexion. Le paramètre enable_thinking n'a aucun effet sur les modèles QwQ .Copy
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation" \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-H "X-DashScope-SSE: enable" \
-d '{
"model": "qwen-plus",
"input":{
"messages":[
{
"role": "user",
"content": "Who are you?"
}
]
},
"parameters":{
"enable_thinking": true,
"incremental_output": true,
"result_format": "message"
}
}'
Réponse
Copy
id:1
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"Hmm","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":14,"input_tokens":11,"output_tokens":3},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:2
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":",","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":15,"input_tokens":11,"output_tokens":4},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:3
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"user","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":16,"input_tokens":11,"output_tokens":5},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:4
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"asked","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":17,"input_tokens":11,"output_tokens":6},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:5
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"\"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":18,"input_tokens":11,"output_tokens":7},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
......
id:358
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"help","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":373,"input_tokens":11,"output_tokens":362},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:359
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":",","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":374,"input_tokens":11,"output_tokens":363},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:360
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"welcome","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":375,"input_tokens":11,"output_tokens":364},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:361
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"anytime","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":376,"input_tokens":11,"output_tokens":365},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:362
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"tell","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":377,"input_tokens":11,"output_tokens":366},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:363
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"me","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:364
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"!","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:365
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"","role":"assistant"},"finish_reason":"stop"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
Mise en production
- Gestion des performances et des ressources : Dans les services backend, le maintien d'une connexion HTTP persistante pour chaque requête en streaming consomme des ressources. Configurez votre service avec une taille de pool de connexions et des valeurs de délai d'attente appropriées. Dans les scénarios de forte concurrence, surveillez l'utilisation des descripteurs de fichiers pour éviter leur épuisement.
-
Rendu côté client : Sur les interfaces web frontales, utilisez les API
ReadableStreametTextDecoderStreampour gérer et restituer fluidement les flux d'événements SSE, offrant ainsi la meilleure expérience utilisateur. -
Surveillance des modèles :
- Métriques clés : Surveillez le Time to First Token (TTFT), la métrique centrale de l'expérience en streaming. Surveillez également le taux d'erreur de l'API et le temps de réponse moyen.
- Alertes : Définissez des alertes pour les taux d'erreur d'API anormaux, en particulier les erreurs 4xx et 5xx.
-
Configuration du proxy Nginx : Si vous utilisez Nginx comme proxy inverse, sa mise en tampon de sortie par défaut (proxy_buffering) rompt la nature en temps réel des réponses en streaming. Pour garantir que les données soient poussées immédiatement vers les clients, désactivez cette fonctionnalité en définissant
proxy_buffering offdans votre fichier de configuration Nginx.
Codes d'erreur
Si l'appel au modèle échoue et renvoie un message d'erreur, consultez les Codes d'erreur pour résoudre le problème.FAQ
Q : Pourquoi n'y a-t-il aucune information d'utilisation dans la réponse ?
R : Le protocole OpenAI ne renvoie pas d'informations d'utilisation par défaut. Définissez le paramètrestream_options pour inclure les informations d'utilisation dans le dernier paquet renvoyé.
Q : L'activation de la sortie en streaming affecte-t-elle la qualité de la réponse du modèle ?
R : Non. Cependant, certains modèles ne prennent en charge que la sortie en streaming, et les appels non streamés peuvent provoquer des erreurs de délai d'attente. Nous recommandons d'utiliser la sortie en streaming.Q : Quelle est la différence entre les appels non streamés et les appels en streaming ?
R : Différences principales :- Limite de délai d'attente : Pour les appels non streamés, le délai d'attente maximal est d'au moins 300 secondes et varie selon la région et le modèle. Si le traitement n'est pas terminé à temps, la requête est interrompue.
- Structure de sortie : Les appels non streamés renvoient la réponse complète (un seul objet JSON) en une fois. Les appels en streaming renvoient progressivement des fragments de données via le protocole SSE, chaque fragment contenant une partie du contenu généré. Le client doit assembler ces fragments.
- Compatibilité des fonctionnalités : Les deux méthodes prennent en charge des fonctionnalités telles que le mode JSON et Function Call, sans différences fonctionnelles.
Q : La sortie en streaming prend-elle en charge le mode JSON (sortie structurée) ?
R : Oui. Définissezstream sur true et response_format sur {"type": "json_object"} dans la requête. Le modèle renverra progressivement des fragments de contenu au format JSON. La sortie finale assemblée sera un JSON valide.