Skip to main content
Alibaba Cloud Model Studio home pageAlibaba Cloud Model Studio
Toolkit/Framework

OpenAI-compatible - Batch (file input)

Alibaba Cloud Model Studio provides an OpenAI-compatible Batch File API. Submit requests in bulk through files. The system processes them asynchronously and returns results when all requests complete or the maximum wait time is reached. Costs are only 50% of real-time calls. Ideal for data analytics, model evaluation, and other large-scale workloads where latency is not critical.

To use the console, see console guide.

Workflow

image

Prerequisites

You can call the Batch File API through the OpenAI SDK (Python, Node.js) or HTTP API.

Scope

  • Singapore
  • China (Beijing)
Supported models: qwen-max, qwen-plus, qwen-flash, qwen-turbo.

Getting started

Before processing formal tasks, test with the batch-test-model. This test model skips inference and returns a fixed success response, allowing you to verify your API call chain and data format.
Limitations of batch-test-model:
  • Your test file must meet Input file requirements. Maximum size: 1 MB. Maximum lines: 100.
  • Concurrency limit: Up to 2 parallel tasks.
  • Cost: The test model does not incur model inference fees.

Step 1: Prepare the input file

Prepare a file named test_model.jsonl with the following content: 
{"custom_id":"1","method":"POST","url":"/v1/chat/ds-test","body":{"model":"batch-test-model","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello! How can I help you?"}]}}
{"custom_id":"2","method":"POST","url":"/v1/chat/ds-test","body":{"model":"batch-test-model","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 2+2?"}]}}
Multimodal models (e.g., qwen-vl-plus) support file URLs and Base64-encoded inputs: 
{"custom_id":"image-url","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"}},{"type":"text","text":"Describe this image."}]}]}}
{"custom_id":"image-base64","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEA8ADwAAD..."}},{"type":"text","text":"Describe this image."}]}]}}

Step 2: Run the code

Select a code snippet for your programming language. Save it in the same directory as your input file and run it. The code handles the full workflow: upload, create task, poll status, and download results.
To customize the file path or other parameters, modify the code as needed.
Reuse an existing file ID: The ID returned after uploading a file (e.g., file-batch-xxx) can be reused. If the input content remains the same, skip re-uploading and directly create a task with the existing ID:
batch = client.batches.create(
    input_file_id="file-batch-xxx",  # Reuse existing file ID, no need to re-upload
    endpoint="/v1/chat/completions",
    completion_window="24h"
)
You can retrieve historical file IDs through the client.files.list(purpose="batch") API to query uploaded Batch file IDs.
OpenAI Python SDK
import os
from pathlib import Path
from openai import OpenAI
import time

# Initialize the client
client = OpenAI(
    # If no environment variable is set, replace the line below with api_key="sk-xxx". But never hard-code in production to reduce leak risk.
    # API keys differ between Singapore and Beijing regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1"  # The base_url for the Model Studio service
)

def upload_file(file_path):
    print(f"Uploading the JSONL file that contains request information...")
    file_object = client.files.create(file=Path(file_path), purpose="batch")
    print(f"File uploaded successfully. File ID: {file_object.id}\n")
    return file_object.id

def create_batch_job(input_file_id):
    print(f"Creating a batch task based on the file ID...")
    # Note: The value of the endpoint parameter here must match the url field in the input file.
    # For the test model (batch-test-model), use /v1/chat/ds-test.
    # For text embedding models, use /v1/embeddings.
    # For other models, use /v1/chat/completions.
    batch = client.batches.create(input_file_id=input_file_id, endpoint="/v1/chat/ds-test", completion_window="24h")
    print(f"Batch task created. Batch task ID: {batch.id}\n")
    return batch.id

def check_job_status(batch_id):
    print(f"Checking the batch task status...")
    batch = client.batches.retrieve(batch_id=batch_id)
    print(f"Batch task status: {batch.status}\n")
    return batch.status

def get_output_id(batch_id):
    print(f"Getting the output file ID for successful requests in the batch task...")
    batch = client.batches.retrieve(batch_id=batch_id)
    print(f"Output file ID: {batch.output_file_id}\n")
    return batch.output_file_id

def get_error_id(batch_id):
    print(f"Getting the output file ID for failed requests in the batch task...")
    batch = client.batches.retrieve(batch_id=batch_id)
    print(f"Error file ID: {batch.error_file_id}\n")
    return batch.error_file_id

def download_results(output_file_id, output_file_path):
    print(f"Printing and downloading the results of successful requests from the batch task...")
    content = client.files.content(output_file_id)
    # Print some content for testing
    print(f"Printing the first 1,000 characters of the successful request results: {content.text[:1000]}...\n")
    # Save the result file locally
    content.write_to_file(output_file_path)
    print(f"The complete output results have been saved to the local output file result.jsonl\n")

def download_errors(error_file_id, error_file_path):
    print(f"Printing and downloading the failure information for requests from the batch task...")
    content = client.files.content(error_file_id)
    # Print some content for testing
    print(f"Printing the first 1,000 characters of the request failure information: {content.text[:1000]}...\n")
    # Save the error information file locally
    content.write_to_file(error_file_path)
    print(f"The complete request failure information has been saved to the local error file error.jsonl\n")

def main():
    # File paths
    input_file_path = "test_model.jsonl"  # You can replace this with your input file path
    output_file_path = "result.jsonl"  # You can replace this with your output file path
    error_file_path = "error.jsonl"  # You can replace this with your error file path
    try:
        # Step 1: Upload the JSONL file containing request information to get the input file ID
        input_file_id = upload_file(input_file_path)
        # Step 2: Create a batch task based on the input file ID
        batch_id = create_batch_job(input_file_id)
        # Step 3: Check the batch task status until it is finished
        status = ""
        while status not in ["completed", "failed", "expired", "cancelled"]:
            status = check_job_status(batch_id)
            print(f"Waiting for the task to complete...")
            time.sleep(10)  # Wait 10 seconds and then query the status again
        # If the task fails, print the error message and exit
        if status == "failed":
            batch = client.batches.retrieve(batch_id)
            print(f"Batch task failed. Error message: {batch.errors}\n")
            print(f"For more information, see Error codes: https://www.alibabacloud.com/help/en/model-studio/developer-reference/error-code
            return
        # Step 4: Download results: If the output file ID is not empty, print the first 1,000 characters of the successful request results and download the complete results to a local output file.
        # If the error file ID is not empty, print the first 1,000 characters of the request failure information and download the complete information to a local error file.
        output_file_id = get_output_id(batch_id)
        if output_file_id:
            download_results(output_file_id, output_file_path)
        error_file_id = get_error_id(batch_id)
        if error_file_id:
            download_errors(error_file_id, error_file_path)
            print(f"For more information, see Error codes: https://www.alibabacloud.com/help/en/model-studio/developer-reference/error-code
    except Exception as e:
        print(f"An error occurred: {e}")
        print(f"For more information, see Error codes: https://www.alibabacloud.com/help/en/model-studio/developer-reference/error-code

if __name__ == "__main__":
    main()

Step 3: Verify test results

After the task succeeds, the result file result.jsonl contains the fixed response {"content":"This is a test result."}: 
{"id":"a2b1ae25-21f4-4d9a-8634-99a29926486c","custom_id":"1","response":{"status_code":200,"request_id":"a2b1ae25-21f4-4d9a-8634-99a29926486c","body":{"created":1743562621,"usage":{"completion_tokens":6,"prompt_tokens":20,"total_tokens":26},"model":"batch-test-model","id":"chatcmpl-bca7295b-67c3-4b1f-8239-d78323bb669f","choices":[{"finish_reason":"stop","index":0,"message":{"content":"This is a test result."}}],"object":"chat.completion"}},"error":null}
{"id":"39b74f09-a902-434f-b9ea-2aaaeebc59e0","custom_id":"2","response":{"status_code":200,"request_id":"39b74f09-a902-434f-b9ea-2aaaeebc59e0","body":{"created":1743562621,"usage":{"completion_tokens":6,"prompt_tokens":20,"total_tokens":26},"model":"batch-test-model","id":"chatcmpl-1e32a8ba-2b69-4dc4-be42-e2897eac9e84","choices":[{"finish_reason":"stop","index":0,"message":{"content":"This is a test result."}}],"object":"chat.completion"}},"error":null}

Run a formal task

Input file requirements

  • Format: UTF-8 encoded JSONL (one independent JSON object per line).
  • Size limits: Maximum 50,000 requests per file, maximum 500 MB.
  • Line limit: Each JSON object must not exceed 6 MB and must fit within the model's context window.
  • Consistency: All requests in the same file must use the same model and the same thinking mode (if applicable).
  • Unique identifier: Each request must include a unique custom_id field within the file. This field is used to match requests with results.
Scenario 1: Text chat Example file content: 
{"custom_id":"1","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-plus","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello!"}]}}
{"custom_id":"2","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-plus","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"What is 2+2?"}]}}
Scenario 2: Image and video understanding Multimodal models (e.g., qwen-vl-plus) support file URLs and Base64-encoded inputs. 
{"custom_id":"image-url","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"}},{"type":"text","text":"Describe this image."}]}]}}
{"custom_id":"image-base64","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEA8ADwAAD..."}},{"type":"text","text":"Describe this image."}]}]}}
{"custom_id":"video-url","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"video","video":"https://example.com/video.mp4"},{"type":"text","text":"Describe this video."}]}]}}
{"custom_id":"video-base64","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"video","video":["data:image/jpeg;base64,{frame1}","data:image/jpeg;base64,{frame2}","data:image/jpeg;base64,{frame3}"]},{"type":"text","text":"Describe this video."}]}]}}
{"custom_id":"multi-image-url","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"https://example.com/image1.jpg"}},{"type":"image_url","image_url":{"url":"https://example.com/image2.jpg"}},{"type":"text","text":"Compare these two images."}]}]}}
{"custom_id":"multi-image-base64","method":"POST","url":"/v1/chat/completions","body":{"model":"qwen-vl-plus","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,{image1_base64}"}},{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,{image2_base64}"}},{"type":"text","text":"Compare these two images."}]}]}}
The Base64 strings in the examples above are truncated. Generate full encodings using the Python code below.
  1. Convert a local file to Base64 encoding:
# Encoding function: Convert a local file to a Base64-encoded string
import base64
def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")

# Replace xxx/eagle.png with the absolute path to your local image
base64_image = encode_image("xxx/eagle.png")
  1. Build a Data URL format: data:[MIME_type];base64,{base64_image};
    1. Replace MIME_type with the actual media type. It must match the MIME Type value (e.g., image/jpeg, image/png);
    2. base64_image is the Base64 string generated in the previous step.
For full details (including file limits, MIME types, and encoding methods), see Pass local files (Base64 encoding or file paths).
JSONL batch generation tool Use this tool to quickly generate JSONL files.

1. Modify the input file

  • In the test_model.jsonl file, set the model parameter to the target model and set the url field:

    Model type

    url

    Text generation/multimodal models

    /v1/chat/completions

    Text embedding models

    /v1/embeddings

  • Or use the "JSONL batch generation tool" above to generate a new file for formal tasks. Ensure the model and url fields are correct.

2. Modify the Getting Started code

  1. Change the input file path to your file name.
  2. Set the endpoint parameter to match the url field in your input file.

3. Run the code and wait for results

When the task completes, successful request results are saved to the local result.jsonl file. If any requests failed, the error details are saved to the error.jsonl file.
  • Successful results (output_file_id): Each line corresponds to one successful request and includes the custom_id and response.
{"id":"3a5c39d5-3981-4e4c-97f2-e0e821893f03","custom_id":"req-001","response":{"status_code":200,"request_id":"3a5c39d5-3981-4e4c-97f2-e0e821893f03","body":{"created":1768306034,"usage":{"completion_tokens":654,"prompt_tokens":14,"total_tokens":668},"model":"qwen-plus","id":"chatcmpl-3a5c39d5-3981-4e4c-97f2-e0e821893f03","choices":[{"finish_reason":"stop","index":0,"message":{"role":"assistant","content":"Hello! Hangzhou West Lake is a famous scenic spot in China, located in the western part of Hangzhou City, Zhejiang Province, hence the name \"West Lake\". It is one of China's top ten scenic spots and a World Cultural Heritage site (listed by UNESCO in 2011). It is renowned worldwide for its beautiful natural scenery and profound cultural heritage.\n\n### I. Natural Landscape\nWest Lake is surrounded by mountains on three sides and borders the city on one side, covering an area of approximately 6.39 square kilometers, shaped like a ruyi scepter with rippling blue waters. The lake is naturally or artificially divided into multiple water areas by Solitary Hill, Bai Causeway, Su Causeway, and Yanggong Causeway, forming a layout of \"one mountain, two pagodas, three islands, and three causeways\".\n\nMain attractions include the following:\n- **Spring Dawn at Su Causeway**: During the Northern Song Dynasty, the great literary figure Su Dongpo, while serving as the prefect of Hangzhou, led the dredging of West Lake and used the excavated silt to build a causeway, later named \"Su Causeway\". In spring, peach blossoms and willows create a picturesque scene.\n- **Lingering Snow on Broken Bridge**: Located at the eastern end of Bai Causeway, this is where the reunion scene from the Legend of the White Snake took place. After snowfall in winter, it is particularly famous for its silver-white appearance.\n- **Leifeng Pagoda at Sunset**: Leifeng Pagoda glows golden under the setting sun and was once one of the \"Ten Scenes of West Lake\".\n- **Three Pools Mirroring the Moon**: On Xiaoyingzhou Island in the lake, there are three stone pagodas. During the Mid-Autumn Festival, lanterns can be lit inside the pagodas, creating a harmonious interplay of moonlight, lamplight, and lake reflections.\n- **Autumn Moon over Calm Lake**: Located at the western end of Bai Causeway, it is an excellent spot for viewing the moon over the lake.\n- **Viewing Fish at Flower Harbor**: Known for viewing flowers and fish, with peonies and koi complementing each other beautifully in the garden.\n\n### II. Cultural History\nWest Lake not only boasts beautiful scenery but also carries rich historical and cultural significance:\n- Since the Tang and Song dynasties, numerous literati such as Bai Juyi, Su Dongpo, Lin Bu, and Yang Wanli have left poems here.\n- Bai Juyi oversaw the construction of \"Bai Causeway\" and dredged West Lake, benefiting the local people.\n- Around West Lake are many historical sites, including Yuewang Temple (commemorating national hero Yue Fei), Lingyin Temple (a millennium-old Buddhist temple), Liuhe Pagoda, and Longjing Village (the origin of Longjing tea, one of China's top ten famous teas).\n\n### III. Cultural Symbolism\nWest Lake is regarded as a representative of \"paradise on earth\" and a model of traditional Chinese landscape aesthetics. It embodies the philosophical concept of \"harmony between heaven and humanity\" by integrating natural beauty with cultural depth. Many poems, paintings, and operas feature West Lake, making it an important symbol of Chinese culture.\n\n### IV. Travel Recommendations\n- Best visiting seasons: Spring (March-May) for peach blossoms and willows, Autumn (September-November) for clear skies and cool weather.\n- Recommended ways: Walking, cycling (along the lakeside greenway), or boating on the lake.\n- Local cuisine: West Lake vinegar fish, Longjing shrimp, Dongpo pork, pian'erchuan noodles.\n\nIn summary, Hangzhou West Lake is not just a natural wonder but also a living cultural museum worth exploring in detail. If you ever visit Hangzhou, don't miss this earthly paradise that is \"equally charming in light or heavy makeup\"."}}],"object":"chat.completion"}},"error":null}
{"id":"628312ba-172c-457d-ba7f-3e5462cc6899","custom_id":"req-002","response":{"status_code":200,"request_id":"628312ba-172c-457d-ba7f-3e5462cc6899","body":{"created":1768306035,"usage":{"completion_tokens":25,"prompt_tokens":18,"total_tokens":43},"model":"qwen-plus","id":"chatcmpl-628312ba-172c-457d-ba7f-3e5462cc6899","choices":[{"finish_reason":"stop","index":0,"message":{"role":"assistant","content":"The spring breeze brushes green willows,\nNight rain nourishes red flowers.\nBird songs fill the forest,\nMountains and rivers share the same beauty."}}],"object":"chat.completion"}},"error":null}
  • Failure details (error_file_id): Contains information about failed requests with line numbers and error reasons. See Error codes for troubleshooting.

Detailed procedure

The Batch API workflow consists of four steps: upload a file, create a task, query task status, and download results.
Upload your JSONL file using the file upload API to obtain a file_id.
When uploading a file, the purpose parameter must be batch .
Reuse an existing file ID: The ID returned after uploading a file (e.g., file-batch-xxx) can be reused. If the input content remains the same, skip re-uploading and directly create a task with the existing ID:
batch = client.batches.create(
    input_file_id="file-batch-xxx",  # Reuse existing file ID, no need to re-upload
    endpoint="/v1/chat/completions",
    completion_window="24h"
)
You can retrieve historical file IDs through the client.files.list(purpose="batch") API to query uploaded Batch file IDs.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)

Request example 

import os
from pathlib import Path
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

# test.jsonl is a local sample file. purpose must be batch.
file_object = client.files.create(file=Path("test.jsonl"), purpose="batch")

print(file_object.model_dump_json())

Response example 

{
    "id": "file-batch-xxx",
    "bytes": 437,
    "created_at": 1742304153,
    "filename": "test.jsonl",
    "object": "file",
    "purpose": "batch",
    "status": "processed",
    "status_details": null
}
Create a batch task using the file ID obtained from the file upload step.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)

Request example 

import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)

batch = client.batches.create(
    input_file_id="file-batch-xxx",  # The ID returned after uploading the file
    endpoint="/v1/chat/completions",  # For the test model batch-test-model, use /v1/chat/ds-test. For text embedding models, use /v1/embeddings. For text generation/multimodal models, use /v1/chat/completions.
    completion_window="24h",
    metadata={'ds_name':"Task name",'ds_description':'Task description'} # Optional metadata field for creating task name and description
)
print(batch)

Input parameters

Field

Type

Method

Required

Description

input_file_id

String

Body

Yes

The input file ID.

Use the file ID returned by the Prepare and upload file API, such as file-batch-xxx.

endpoint

String

Body

Yes

The API access path. Must match the url field in the input file.

  • For the test model batch-test-model, enter /v1/chat/ds-test

  • For other models, enter /v1/chat/completions

completion_window

String

Body

Yes

Maximum wait time. Range: 24h-336h, integers only.

Units: "h" or "d" (e.g., "24h" or "14d").

metadata

Map

Body

No

Extended metadata for the task, specified as key-value pairs.

metadata.ds_name

String

Body

No

Task name.

Example: "ds_name": "Batch task"

Maximum length: 100 characters.

If specified multiple times, the last value takes effect.

metadata.ds_description

String

Body

No

Task description.

Example: "ds_description": "Batch inference task test"

Maximum length: 200 characters.

If specified multiple times, the last value takes effect.

Response example 

{
    "id": "batch_xxx",
    "object": "batch",
    "endpoint": "/v1/chat/completions",
    "errors": null,
    "input_file_id": "file-batch-xxx",
    "completion_window": "24h",
    "status": "validating",
    "output_file_id": null,
    "error_file_id": null,
    "created_at": 1742367779,
    "in_progress_at": null,
    "expires_at": null,
    "finalizing_at": null,
    "completed_at": null,
    "failed_at": null,
    "expired_at": null,
    "cancelling_at": null,
    "cancelled_at": null,
    "request_counts": {
        "total": 0,
        "completed": 0,
        "failed": 0
    },
    "metadata": {
        "ds_name": "Task name",
        "ds_description": "Task description"
    }
}

Response parameters

Field

Type

Description

id

String

The batch task ID.

object

String

Fixed value: batch.

endpoint

String

The API access path.

errors

Map

Error information.

input_file_id

String

Input file ID .

completion_window

String

Maximum wait time. Range: 24h-336h, integers only.

Units: "h" or "d" (e.g., "24h" or "14d").

status

String

Task status: validating, failed, in_progress, finalizing, completed, expired, cancelling, cancelled.

output_file_id

String

File ID for successful request results.

error_file_id

String

File ID for failed request results.

created_at

Integer

Unix timestamp (seconds) when the task was created.

in_progress_at

Integer

Unix timestamp (seconds) when the task started processing.

expires_at

Integer

Unix timestamp (seconds) when the task starts timing out.

finalizing_at

Integer

Unix timestamp (seconds) when the task last started.

completed_at

Integer

Unix timestamp (seconds) when the task completed.

failed_at

Integer

Unix timestamp (seconds) when the task failed.

expired_at

Integer

Unix timestamp (seconds) when the task expired.

cancelling_at

Integer

Unix timestamp (seconds) when the task entered the cancelling state.

cancelled_at

Integer

Unix timestamp (seconds) when the task was cancelled.

request_counts

Map

Request counts by state.

metadata

Map

Additional metadata as key-value pairs.

metadata.ds_name

String

Task name.

metadata.ds_description

String

Task description.

After creating a task, use the following APIs to query its status, list historical tasks, or cancel an ongoing task.
Query a batch task by its ID. Only tasks created within the past 30 days can be queried.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)

Request example 

import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
batch = client.batches.retrieve("batch_id")  # Replace batch_id with the batch task ID
print(batch)

Response example

A successful query returns detailed batch task information. The following is a response example for a task with completed status:
{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "file-batch_output-xyz789",
  "error_file_id": "file-batch_error-xyz789",
  "created_at": 1711402400,
  "in_progress_at": 1711402450,
  "expires_at": 1711488800,
  "finalizing_at": 1711405000,
  "completed_at": 1711406000,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": {
    "total": 100,
    "completed": 95,
    "failed": 5
  },
  "metadata": {
    "customer_id": "user_123456789",
    "batch_description": "Nightly eval job"
  }
}
For field descriptions, see the table below.

Field

Type

Description

id

String

Batch task ID.

status

String

Task status. Possible values:

  • validating

  • in_progress

  • finalizing

  • completed

  • failed

  • expired

  • cancelling

  • cancelled

output_file_id

String

ID of the output file containing successful results. Generated after task completion.

error_file_id

String

ID of the error file containing failed request details. Generated after task completion if any requests failed.

request_counts

Object

Request count statistics containing total, completed, and failed counts.

Use the batches.list() method to retrieve the list of batch tasks. Use pagination to retrieve the full task list.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)

Request example 

import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
batches = client.batches.list(after="batch_xxx", limit=2,extra_query={'ds_name':'Task name','input_file_ids':'file-batch-xxx,file-batch-xxx','status':'completed,expired','create_after':'20250304000000','create_before':'20250306123000'})
print(batches)

Input parameters

Field

Type

Method

Required

Description

after

String

Query

No

Cursor for pagination. Set this to the last task ID from the previous page.

limit

Integer

Query

No

Number of tasks per page. Range: [1, 100]. Default: 20.

ds_name

String

Query

No

Fuzzy match by task name.

input_file_ids

String

Query

No

Filter by file IDs. Specify multiple IDs separated by commas (up to 20).

status

String

Query

No

Filter by task status. Specify multiple statuses separated by commas.

create_after

String

Query

No

Filter tasks created after this time. Format: yyyyMMddHHmmss.

create_before

String

Query

No

Filter tasks created before this time. Format: yyyyMMddHHmmss.

Response example 

{
  "object": "list",
  "data": [
    {
      "id": "batch_xxx",
      "object": "batch",
      "endpoint": "/v1/chat/completions",
      "errors": null,
      "input_file_id": "file-batch-xxx",
      "completion_window": "24h",
      "status": "completed",
      "output_file_id": "file-batch_output-xxx",
      "error_file_id": null,
      "created_at": 1722234109,
      "in_progress_at": 1722234109,
      "expires_at": null,
      "finalizing_at": 1722234165,
      "completed_at": 1722234165,
      "failed_at": null,
      "expired_at": null,
      "cancelling_at": null,
      "cancelled_at": null,
      "request_counts": {
        "total": 100,
        "completed": 95,
        "failed": 5
      },
      "metadata": {}
    },
    { ... }
  ],
  "first_id": "batch_xxx",
  "last_id": "batch_xxx",
  "has_more": true
}

Response parameters

Field

Type

Description

object

String

Object type. Fixed value: list.

data

Array

Array of batch task objects. See the response parameters for creating a batch task.

first_id

String

ID of the first batch task on the current page.

last_id

String

ID of the last batch task on the current page.

has_more

Boolean

Whether additional pages are available.

Cancel a task that is in progress or queued. After a successful call, the task status changes to cancelling and then to cancelled. You are still billed for requests that completed before the cancellation takes full effect.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)

Request example 

import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
batch = client.batches.cancel("batch_id")  # Replace batch_id with the batch task ID
print(batch)

Response example

After you successfully cancel a task, the API returns detailed batch task information. The following is a response example for a task in cancelling status:
{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "cancelling",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1711402400,
  "in_progress_at": 1711402450,
  "expires_at": 1711488800,
  "finalizing_at": null,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": 1711403000,
  "cancelled_at": null,
  "request_counts": {
    "total": 100,
    "completed": 23,
    "failed": 1
  },
  "metadata": null
}
After you cancel a task, the status first changes to cancelling while the system waits for currently executing requests to complete. The status eventually changes to cancelled . Results from completed requests are still saved in the output file.
After a task completes, result files (output_file_id) and error files (error_file_id) may be generated. Download both file types using the same file download API.You can download only files whose file_id starts with file-batch_output.
  • OpenAI Python SDK
  • OpenAI Node.js SDK
  • Java (HTTP)
  • curl (HTTP)
Use the content method to retrieve the batch task result file content and use the write_to_file method to save it locally.

Request example 

import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Singapore region's base_url. For Beijing, use: https://dashscope.aliyuncs.com/compatible-mode/v1 and update the API key.
    # Note: When switching regions, also update the API key accordingly.
    base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
content = client.files.content(file_id="file-batch_output-xxx")
# Print result file content
print(content.text)
# Save result file locally
content.write_to_file("result.jsonl")

Response example 

{"id":"c308ef7f-xxx","custom_id":"1","response":{"status_code":200,"request_id":"c308ef7f-0824-9c46-96eb-73566f062426","body":{"created":1742303743,"usage":{"completion_tokens":35,"prompt_tokens":26,"total_tokens":61},"model":"qwen-plus","id":"chatcmpl-c308ef7f-0824-9c46-96eb-73566f062426","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! Of course. Whether you need information, learning materials, problem-solving methods, or any other help, I am here to support you. Please tell me what you need help with."}}],"object":"chat.completion"}},"error":null}
{"id":"73291560-xxx","custom_id":"2","response":{"status_code":200,"request_id":"73291560-7616-97bf-87f2-7d747bbe84fd","body":{"created":1742303743,"usage":{"completion_tokens":7,"prompt_tokens":26,"total_tokens":33},"model":"qwen-plus","id":"chatcmpl-73291560-7616-97bf-87f2-7d747bbe84fd","choices":[{"finish_reason":"stop","index":0,"message":{"content":"2+2 equals 4."}}],"object":"chat.completion"}},"error":null}

Response example

Single response example:
{
    "id": "c308ef7f-xxx",
    "custom_id": "1",
    "response": {
        "status_code": 200,
        "request_id": "c308ef7f-0824-9c46-96eb-73566f062426",
        "body": {
            "created": 1742303743,
            "usage": {
                "completion_tokens": 35,
                "prompt_tokens": 26,
                "total_tokens": 61
            },
            "model": "qwen-plus",
            "id": "chatcmpl-c308ef7f-0824-9c46-96eb-73566f062426",
            "choices": [
                {
                    "finish_reason": "stop",
                    "index": 0,
                    "message": {
                        "content": "Hello! Of course. Whether you need information, learning materials, problem-solving methods, or any other help, I am here to support you. Please tell me what you need help with."
                    }
                }
            ],
            "object": "chat.completion"
        }
    },
    "error": null
}

Response parameters

Field

Type

Description

id

String

The request ID.

custom_id

String

The user-defined request identifier.

response

Object

The request result.

status_code

Integer

HTTP status code. 200 indicates success.

request_id

String

Server-generated unique ID for this request.

completion_tokens

Integer

Number of tokens in the model-generated response.

prompt_tokens

Integer

Number of tokens in the input content (prompt) sent to the model.

total_tokens

Integer

Total number of tokens used by this request.

model

String

Name of the model used for this request.

error

Object

The error object. Returns null on success. On failure, contains the error code and detailed message.

error.code

String

Error line and reason information. See Error codes for troubleshooting.

error.message

String

Error message.

Advanced features

Set completion notifications

For long-running tasks, use asynchronous notifications instead of polling to reduce resource consumption.
Completion notification is only supported in the Beijing region.
  • Callback: Specify a publicly accessible URL when creating the task.
  • EventBridge message queue: Deeply integrated with the Alibaba Cloud ecosystem. No public IP required.
When creating a task, specify a publicly accessible URL via metadata. After the task completes, the system sends a POST request containing the task status to the specified URL:
  • OpenAI Python SDK
  • curl (HTTP)
import os
from openai import OpenAI

client = OpenAI(
    # If no environment variable is set, use api_key="sk-xxx" (not in production - risk of leaks).
    # API keys differ between regions.
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Beijing region's base_url. For Singapore, use: https://dashscope-intl.aliyuncs.com/compatible-mode/v1 and update the API key.
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

batch = client.batches.create(
    input_file_id="file-batch-xxx",  # The ID returned after uploading the file
    endpoint="/v1/chat/completions",  # For text embedding models, enter "/v1/embeddings". For the test model batch-test-model, enter /v1/chat/ds-test. For other models, enter /v1/chat/completions.
    completion_window="24h",
    metadata={
            "ds_batch_finish_callback": "https://xxx/xxx"
          }
)
print(batch)
This method requires no public IP and is suitable for complex scenarios that need integration with services such as Function Compute or RocketMQ.When a batch task completes, the system sends an event to Alibaba Cloud EventBridge. Configure EventBridge rules to listen for this event and route it to a specified target.
  • Event source (Source): acs.dashscope
  • Event type (Type): dashscope:System:BatchTaskFinish
Reference: Route events to Message Queue for RocketMQ.

Going live

  • File management
    • Periodically delete unnecessary files using the OpenAI File delete API to avoid reaching storage limits (10,000 files or 100 GB).
    • Store large files in OSS instead of uploading directly.
  • Task monitoring
    • Use Callback or EventBridge asynchronous notifications.
    • If polling is required, set the interval to more than 1 minute and use an exponential backoff strategy.
  • Error handling
    • Implement handling for network errors, API errors, and other exceptions.
    • Download and analyze error details from error_file_id.
    • For common error codes, see Error codes.
  • Cost optimization
    • Consolidate small tasks into a single batch.
    • Set completion_window appropriately to allow greater scheduling flexibility.

Utility tools

If your data is in a CSV file (first column: ID, second column: content), use this script to generate a JSONL batch input file.
To customize the file path or other parameters, modify the code as needed. 
import csv
import json
def messages_builder_example(content):
    messages = [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": content}]
    return messages

with open("input_demo.csv", "r") as fin:
    with open("input_demo.jsonl", 'w', encoding='utf-8') as fout:
        csvreader = csv.reader(fin)
        for row in csvreader:
            body = {"model": "qwen-turbo", "messages": messages_builder_example(row[1])}
            # When calling a text embedding model, set the url value to "/v1/embeddings". For other models, set it to /v1/chat/completions.
            request = {"custom_id": row[0], "method": "POST", "url": "/v1/chat/completions", "body": body}
            fout.write(json.dumps(request, separators=(',', ':'), ensure_ascii=False) + "\n")
Use this script to convert result.jsonl into result.csv for analysis in Excel.
To customize the file path or other parameters, modify the code as needed. 
import json
import csv
columns = ["custom_id",
           "model",
           "request_id",
           "status_code",
           "error_code",
           "error_message",
           "created",
           "content",
           "usage"]

def dict_get_string(dict_obj, path):
    obj = dict_obj
    try:
        for element in path:
            obj = obj[element]
        return obj
    except:
        return None

with open("result.jsonl", "r") as fin:
    with open("result.csv", 'w', encoding='utf-8') as fout:
        rows = [columns]
        for line in fin:
            request_result = json.loads(line)
            row = [dict_get_string(request_result, ["custom_id"]),
                   dict_get_string(request_result, ["response", "body", "model"]),
                   dict_get_string(request_result, ["response", "request_id"]),
                   dict_get_string(request_result, ["response", "status_code"]),
                   dict_get_string(request_result, ["error", "error_code"]),
                   dict_get_string(request_result, ["error", "error_message"]),
                   dict_get_string(request_result, ["response", "body", "created"]),
                   dict_get_string(request_result, ["response", "body", "choices", 0, "message", "content"]),
                   dict_get_string(request_result, ["response", "body", "usage"])]
            rows.append(row)
        writer = csv.writer(fout)
        writer.writerows(rows)
Fix garbled text in Excel
  • Use a text editor (such as Sublime Text) to convert the CSV file encoding to GBK, then open it in Excel.
  • Alternatively, create a new Excel file and specify UTF-8 encoding when importing the data.

Rate limits

API

Rate limit (per Alibaba Cloud account)

Create task

1,000 calls/minute; up to 1,000 concurrent tasks

Query task

1,000 calls/minute

Query task list

100 calls/minute

Cancel task

1,000 calls/minute

Billing

  • Unit price: The input and output tokens for all successful requests are charged at 50% of the real-time inference price for the corresponding model. For more information, see Model list.
  • Billing scope:
    • Only requests successfully executed within a task are billed.
    • Requests that fail because of file parsing errors, task execution failures, or row-level errors do not incur charges.
    • For canceled tasks, requests successfully completed before the cancellation are still billed as normal.
  • Batch inference is a separate billing item. It supports AI general-purpose savings plan, but not discounts, such as subscription (other savings plans) or free quotas for new users. It also does not support features such as context cache.
  • Some models, such as qwen3.5-plus and qwen3.5-flash, have thinking mode enabled by default. This mode generates additional thinking tokens, which are billed at the output token price and increase costs. To control costs, set the enable_thinking parameter based on task complexity. For more information, see Deep thinking.

Error codes

If a request fails and returns an error message, see Error codes for a solution.

FAQ

  1. How do I choose between Batch Chat and Batch File? Use Batch File when you need to process a large file containing many requests asynchronously. Use Batch Chat when your business logic requires submitting many independent conversation requests synchronously with high concurrency.
  2. How is the Batch File API billed? Do I need to purchase a separate package? Batch uses pay-as-you-go billing based on tokens consumed by successful requests. No separate resource package is required.
  3. Are submitted batch files executed in order? No. The system uses dynamic scheduling based on compute load and does not guarantee execution order. Tasks may be delayed when resources are constrained.
  4. How long does it take to complete a submitted batch file? Execution time depends on system resources and task scale. If a task does not complete within the completion_window, it expires. Unprocessed requests in expired tasks are not executed and do not incur charges. Scenario recommendations: Use real-time calls for scenarios requiring strict real-time model inference. Use batch calls for large-scale data processing scenarios that can tolerate delay.
Alibaba Cloud Model Studio home pageAlibaba Cloud Model Studio
API Usage
Text Generation
Image Generation
  • FAQ
Video Generation
Audio
Real-time Multimodal
Realtime API
Text Embedding
More Models
Toolkit/Framework
Model Production
File Management
More