Skip to main content
Alibaba Cloud Model Studio home pageAlibaba Cloud Model Studio
Multimodal embedding

Multimodal embedding API

Multimodal embedding models convert text, images, and videos into embeddings in a shared semantic space to enable cross-modal retrieval, content classification, and similarity search.

Core capabilities

  • Cross-modal retrieval: Perform semantic searches across different content types, such as text-to-image, image-to-video, or image-to-image.
  • Semantic similarity: Measure the semantic similarity between different content types in a unified embedding space.
  • Content classification and clustering: Group, label, and cluster content based on semantic embeddings.
Key feature: Embeddings for all modalities (text, images, and video) share the same semantic space, enabling direct cross-modal matching and comparison using methods such as cosine similarity. See text and multimodal embedding for details on model selection and usage.
This model service is only available in the China (Beijing) region. To call the service, use an API key from this region.

Embedding types

The multimodal embedding model supports two methods for generating embeddings:
  • Multimodal independent embedding: Generates a separate embedding for each input, such as text, an image, a video, or multiple images, within the contents. For example, an input of one text string and one image returns two independent embeddings. This is ideal for comparing individual items, such as in image-to-image or text-to-image searches.
  • Multimodal fused embedding: Fuses all inputs in contents into a single embedding to achieve a unified cross-modal semantic representation. This is suitable for scenarios that require a holistic understanding of multimodal content, such as fusing a product image and its description text into a unified representation for retrieval. For qwen3-vl-embedding, you enable fusion by setting enable_fusion=true. The fused embedding supports the following combinations:
    • Text and image fusion
    • Text and video fusion
    • Fusing multiple images with text (by passing multiple image entries)
    • Fusion of images, video, and text
qwen2.5-vl-embedding supports only fused embeddings, not independent embeddings. tongyi-embedding-vision-plus and tongyi-embedding-vision-flash support only independent embeddings.
For model introductions, selection guidance, and usage instructions, see Text and multimodal embedding.

Model overview

  • Singapore
  • China (Beijing)

Model

Embedding dimensions

Text length limit

Image size limit

Video size limit

Price (per 1M tokens)

Free quota(Note)

tongyi-embedding-vision-plus

1152

1,024 token

Up to 3 MB per image. Supports up to 8 images.

Up to 10 MB per video file

Image/Video: $0.09

Text: $0.09

1 million token

This free quota is valid for 90 days from the date of Model Studio activation, model release, or application approval, whichever is later.

tongyi-embedding-vision-flash

768

Image/Video: $0.03

Text: $0.09

Input formats and usage limits

Fused multimodal model
ModelTextImageVideoRequest limit
qwen3-vl-embeddingSupports 33 major languages, such as Chinese, English, Japanese, Korean, French, and German.
Chinese, Japanese, Korean, Indonesian, Vietnamese, Thai, English, French, German, Russian, Portuguese, Spanish, Italian, Swedish, Danish, Czech, Norwegian, Dutch, Finnish, Turkish, Polish, Swahili, Romanian, Serbian, Greek, Kazakh, Uzbek, Cebuano, Arabic, Urdu, Persian, Hindi/Devanagari, and Hebrew.
JPEG, PNG, WEBP, BMP, TIFF, ICO, DIB, ICNS, and SGI (URL or Base64 supported)MP4, AVI, and MOV (URL only)The total number of content elements in a single request cannot exceed 20. The number of images cannot exceed 5. Images, text, and videos share this limit.
Independent multimodal model
ModelTextImageVideoRequest limit
tongyi-embedding-vision-plusChinese/EnglishJPG, PNG, and BMP (URL or Base64 supported)MP4, MPEG, AVI, MOV, MPG, WEBM, FLV, and MKV (URL only)No limit on the number of content elements. The total number of input tokens must not exceed the token limit.
tongyi-embedding-vision-flash
multimodal-embedding-v1The total number of content elements in a single request cannot exceed 20. A request can contain a maximum of 1 image, 1 video, and 20 text entries. These items share the total limit.
All models accept text, image, and video inputs, individually or in combination. tongyi-embedding-vision-plus, tongyi-embedding-vision-flash models also support multi_images for image sequences.

Model capabilities

Model

Default dimension

Vector type

Supported inputs

Description

qwen3-vl-embedding

2560

Independent / Fusion

text, image, video, multiple images

Fusion mode, enabled with the enable_fusion parameter, combines multimodal inputs into a single vector.

tongyi-embedding-vision-plus

1152

Independent only

text, image, video, multi_images

Supports multi_images sequences (up to 8 images).

tongyi-embedding-vision-flash

768

multimodal-embedding-v1

1024

text, image, video

The vector dimension is fixed at 1,024 and cannot be configured.

Prerequisites

Obtain an API key and export the API key as an environment variable. If you use an SDK to make calls, install the DashScope SDK.

HTTP call

POST https://dashscope.aliyuncs.com/api/v1/services/embeddings/multimodal-embedding/multimodal-embedding

Request

  • Multimodal independent embedding
  • Multimodal fused embedding
The following example uses the tongyi-embedding-vision-plus model to generate an independent embedding for each input. You can replace the model name with another supported model. The multi_images type is supported only by tongyi-embedding-vision-plus and tongyi-embedding-vision-flash. The qwen3-vl-embedding model also supports a fused embedding mode, which you can enable by setting enable_fusion=true. For details, see the "Multimodal fused embedding" tab. 
curl --silent --location --request POST 'https://dashscope.aliyuncs.com/api/v1/services/embeddings/multimodal-embedding/multimodal-embedding' \
    --header "Authorization: Bearer $DASHSCOPE_API_KEY" \
    --header 'Content-Type: application/json' \
    --data '{
        "model": "tongyi-embedding-vision-plus",
        "input": {
            "contents": [
                {"text": "Multimodal embedding model"},
                {"image": "https://img.alicdn.com/imgextra/i3/O1CN01rdstgY1uiZWt8gqSL_!!6000000006071-0-tps-1970-356.jpg"},
                {"video": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250107/lbcemt/new+video.mp4"},
                {"multi_images": [
                    "https://img.alicdn.com/imgextra/i2/O1CN019eO00F1HDdlU4Syj5_!!6000000000724-2-tps-2476-1158.png",
                    "https://img.alicdn.com/imgextra/i2/O1CN01dSYhpw1nSoamp31CD_!!6000000005089-2-tps-1765-1639.png"
                    ]
                  }
            ]
        }
    }'

Request headers

Content-Type string (Required)The content type of the request. Must be application/json.
Authorization string (Required)Authenticates the request with a Model Studio API key. Example: Bearer sk-xxxx.

Request body

model string(required)The model name. Select a model from the Model overview.
input object(required)The input content.

Properties

contentsarray(required)The content items to process. Each item is a dictionary or string that specifies the content type and value in the format {"modality_type": "input_string_or_image/video_url"}. The supported modality types are text, image, video, and multi_images.
The qwen3-vl-embedding model supports both fused and independent embedding generation. To generate a fused embedding, add the boolean field enable_fusion and set it to true. The qwen2.5-vl-embedding model supports only fused embeddings.
  • Text: The key is text, and the value is a string. You can also pass the string directly without a dictionary.
  • Image: Use the image key. The value can be a public URL or a Base64-encoded Data URI. The Base64 format is data:image/{format};base64,{data}, where {format} is the image format, such as jpeg or png, and {data} is the Base64-encoded string.
  • Multiple images: This type is supported only by the tongyi-embedding-vision-plus, tongyi-embedding-vision-flash models. The key is multi_images, and the value is a list of images. Each item in the list is an image that must follow the format described above.
  • Video: The key is video. The value must be a publicly accessible URL.
parameters object (optional)Embedding processing parameters. For HTTP calls, you must wrap these parameters in the parameters object. For SDK calls, you can use these parameters directly.

Properties

output_type string (optional)The format for the output embedding representation. Currently, only dense is supported.dimension integer (optional)The output embedding dimension. Supported values vary by model:
  • qwen3-vl-embedding: Supports 2560, 2048, 1536, 1024, 768, 512, and 256. The default is 2560.
  • tongyi-embedding-vision-plus: Does not support this parameter. Returns a fixed 1152-dimension embedding.
  • tongyi-embedding-vision-flash: Does not support this parameter. Returns a fixed 768-dimension embedding.
  • multimodal-embedding-v1: Does not support this parameter. Returns a fixed 1024-dimension embedding.
fps float (optional)The video frame sampling rate. A smaller value extracts fewer frames. The valid range is [0, 1], and the default is 1.0.instruct string (optional)A custom task description to help the model understand the query's intent. English instructions are recommended and can improve performance by 1% to 5%.enable_fusion bool (optional)Specifies whether to generate a fused embedding. This parameter is supported only by the qwen3-vl-embedding model. When set to true, all multimodal content in the contents array is fused into a single embedding. The default value is false, which generates an independent embedding for each modality. Fused embeddings support combinations such as text and image, text and video, multiple images and text (by passing multiple image items), and a mix of image, video, and text. This is suitable for retrieval scenarios that require a comprehensive understanding of multimodal content.

Response

  • Successful response
  • Error response
{
    "output": {
        "embeddings": [
            {
                "index": 0,
                "embedding": [
                    -0.026611328125,
                    -0.016571044921875,
                    -0.02227783203125,
                    ...
                ],
                "type": "text"
            },
            {
                "index": 1,
                "embedding": [
                    0.051544189453125,
                    0.007717132568359375,
                    0.026611328125,
                    ...
                ],
                "type": "image"
            },
            {
                "index": 2,
                "embedding": [
                    -0.0217437744140625,
                    -0.016448974609375,
                    0.040679931640625,
                    ...
                ],
                "type": "video"
            }
        ]
    },
    "usage": {
        "input_tokens": 10,
        "input_tokens_details": {
            "image_tokens": 896,
            "text_tokens": 7
        },
        "output_tokens": 3,
        "total_tokens": 906
    },
    "request_id": "1fff9502-a6c5-9472-9ee1-73930fdd04c5"
}
The usage field varies by model. See the following descriptions:
  • tongyi-embedding-vision-* series models: Return input_tokens (sum of text and image tokens), input_tokens_details (including image_tokens and text_tokens), output_tokens, and total_tokens. The response example above is for this type of model.
  • qwen3-vl-embedding: Returns only input_tokens (text tokens only, including system template tokens), image_tokens, and total_tokens (= input_tokens + image_tokens). Does not return input_tokens_details or output_tokens. Example:
{
    "usage": {
        "input_tokens": 43,
        "image_tokens": 1247,
        "total_tokens": 1290
    }
}
  • qwen2.5-vl-embedding: Returns only input_tokens and image_tokens. Does not return total_tokens, input_tokens_details, or output_tokens.
  • multimodal-embedding-v1: Returns input_tokens, image_tokens, image_count, and duration. Does not return total_tokens, input_tokens_details, or output_tokens.
output objectTask output.

Properties

embeddings arrayA list of the resulting embeddings, where each object corresponds to an input element.

Properties

index intThe index of the result in the input list.embedding arrayThe dimension of the generated array of embeddings depends on the model and the dimension parameter.type stringThe input type for this result. text, image, video, and multi_images correspond to text, image, video, and multi-image inputs, respectively. Special types include: fusion is the type returned by the qwen3-vl-embedding model in fused embedding mode; vl is the type returned by the qwen3-vl-embedding model in independent embedding mode.
request_id stringUnique request identifier for tracing and troubleshooting.
code stringError code. Returned only for failed requests. See Error codes.
message stringDetailed error message. Returned only for failed requests. See Error codes.
usage objectStatistics about token usage.

Properties

input_tokens intThe number of tokens in the input content for the current request. For the qwen3-vl-embedding and qwen2.5-vl-embedding models, this value includes only text tokens (including system template tokens) and does not include image or video tokens. For the tongyi-embedding-vision-* series models, this value includes the total number of text, image, and video tokens.input_tokens_details objectA detailed breakdown of input tokens. This field is returned only by the tongyi-embedding-vision-* series models. It is not returned by the qwen3-vl-embedding, qwen2.5-vl-embedding, or multimodal-embedding-v1 models.

Properties

image_tokens intThe number of tokens for the input images or videos.text_tokens intThe number of tokens for the input text.
output_tokens intThe number of tokens in the output for the current request. This field is returned only by the tongyi-embedding-vision-* series models.total_tokens intThe total number of input and output tokens. This field is returned by the qwen3-vl-embedding and tongyi-embedding-vision-* models, but not by the qwen2.5-vl-embedding or multimodal-embedding-v1 models. For the qwen3-vl-embedding model, total_tokens = input_tokens + image_tokens.image_tokens intThe number of tokens for the input images or videos in the current request. The system samples frames from input videos, with the maximum number of frames controlled by the system configuration, and then calculates the tokens based on the processed result. This field is returned as a top-level field only by the qwen3-vl-embedding, qwen2.5-vl-embedding, and multimodal-embedding-v1 models. For the tongyi-embedding-vision-* series models, the image token count is included in input_tokens_details.image_tokens.image_count intThe number of images in the input for the current request. This field is returned only by the multimodal-embedding-v1 model.duration intThe duration of the input video in seconds. This field is returned only by the multimodal-embedding-v1 model.

SDK usage

The SDK's input parameter maps to input.contents in the HTTP request body, but their structures are different.

Code examples

  • Image embedding
  • Video embedding
  • Text embedding
  • Fused embedding
  • Multi-image fused embedding
  • 2026-03-06 snapshot version
  • Image URL
  • Local image
import dashscope
import json
from http import HTTPStatus
# Replace with your image URL.
image = "https://dashscope.oss-cn-beijing.aliyuncs.com/images/256_1.png"
input = [{'image': image}]
# Call the model API.
resp = dashscope.MultiModalEmbedding.call(
    model="tongyi-embedding-vision-plus",
    input=input
)

if resp.status_code == HTTPStatus.OK:
    result = {
        "status_code": resp.status_code,
        "request_id": getattr(resp, "request_id", ""),
        "code": getattr(resp, "code", ""),
        "message": getattr(resp, "message", ""),
        "output": resp.output,
        "usage": resp.usage
    }
    print(json.dumps(result, ensure_ascii=False, indent=4))

Output example 

{
    "status_code": 200,
    "request_id": "40532987-ba72-42aa-a178-bb58b52fb7f3",
    "code": "",
    "message": "",
    "output": {
        "embeddings": [
            {
                "index": 0,
                "embedding": [
                    -0.009490966796875,
                    -0.024871826171875,
                    -0.031280517578125,
                    ...
                ],
                "type": "text"
            }
        ]
    },
    "usage": {
        "input_tokens": 10,
        "input_tokens_details": {
            "image_tokens": 0,
            "text_tokens": 10
        },
        "output_tokens": 1,
        "total_tokens": 11
    }
}

Error codes

If the model call fails and returns an error message, see Error codes for resolution.
Alibaba Cloud Model Studio home pageAlibaba Cloud Model Studio
API Usage
Text Generation
Image Generation
  • FAQ
Video Generation
Audio
Real-time Multimodal
Realtime API
Text Embedding
More Models
Toolkit/Framework
Model Production
File Management
More