Provisioned Throughput Long Input and Cache - Alibaba Cloud Model Studio

Overview

PTU deployments support long-input requests (up to 200K tokens for some models) and prefix caching. Tiered capacity coefficients and cache discounts provide flexible quota consumption management. Core capabilities:

Long-input support: Some models support inputs exceeding 32K tokens. Tokens beyond the threshold are consumed at higher tiered coefficients when converted to TPM (tokens per minute). For details, see Quota consumption rules.
Prefix cache discount: Some models support prefix caching. Input tokens that hit the cache consume quota at a discounted coefficient (the specific discount varies by model), reducing quota consumption in multi-turn conversations and repeated prefix scenarios.
Automatic pay-as-you-go fallback: When PTU quota is exceeded or the input exceeds the model limit (128K for Qwen / 64K for DeepSeek), the request automatically falls back to pay-as-you-go billing. No changes to your calling code are required.

After automatic pay-as-you-go fallback, charges are calculated at the model's pay-as-you-go unit price. Use the Provisioned Throughput Quota Calculator to plan your PTU quota appropriately and avoid unexpected charges.

This feature is commonly used in long-document analysis (contracts, research report summaries) and multi-turn conversations (customer service, coding assistants) where input exceeds 32K tokens. For basic concepts and purchasing methods for PTU deployments, see Introduction to model deployment. For how prefix caching works, see Context Cache.

Quota consumption rules

Long-input tiered coefficients and cache discounts vary by model. The following table lists the parameters for currently supported models:

Model	Maximum input length	Cache discount	Long-input tiered coefficients
qwen3.8-Max	1 Million	0.125 (cache-hit tokens consume 12.5% capacity)	No tiers (1.0)
qwen3.7-plus-2026-05-26	1 Million	0.2 (cache-hit tokens consume 20% capacity)	No tiers (1.0)
qwen3.7-flash-2026-07-15	1 Million	0.2 (cache-hit tokens consume 20% capacity)	[0, 32K): input 1.0 / output 1.0 (32K, 256K]: input 3.0 / output 3.0 (256K, 1Million]: input 6.0 / output 6.0
glm-5.2	1 Million	0.25 (cache-hit tokens consume 25% capacity)	No tiers (1.0)
glm-5.1	200K	0.2 (cache-hit tokens consume 20% capacity)	[0, 32K): input 1.0 / output 1.0 (32K, 200K]: input 1.33 / output 1.17
deepseek-v4-pro	256K	0.08 (cache-hit tokens consume 8% capacity)	No tiers (1.0)
deepseek-v4-flash-0731	1 Million	0.2 (cache-hit tokens consume 20% capacity)	No tiers (1.0)
Other models	Refer to the console	Refer to the console	No tiers (1.0)

Calculation examples (glm-5.1)

Scenario 1: Short input (10K tokens, no cache)
  Input consumption: 10K × 1.0 = 10 KTPM

Scenario 2: Long input (50K tokens, no cache)
  Input consumption: 32K × 1.0 + 18K × 1.33 = 55.94 KTPM
  Output consumption (assuming 1K tokens): 1K × 1.17 = 1.17 KTPM

Scenario 3: Long input + cache hit (50K tokens, first 30K hit cache)
  Cached input (first 30K, within [0,32K) tier):
    30K × 1.0 × 0.2 = 6 KTPM
  Non-cached input (last 20K):
    2K × 1.0 + 18K × 1.33 = 25.94 KTPM
  Total input = 31.940 KTPM (43% savings compared to no cache)

Estimating quota with the Provisioned Throughput Quota Calculator

Use the calculator before creating or scaling a deployment to evaluate quota requirements for long-input scenarios. This helps avoid requests falling back to pay-as-you-go billing due to insufficient quota. Refer to the console for actual purchase limits.

Prerequisites: You have activated Model Studio and have PTU deployment permissions. Log on to the Model Studio console, go to the Model Deployment > Create Deployment page (or click Scale Up on an existing deployment's product page), select a deployable PTU (Provisioned Throughput) model, and expand the Provisioned Throughput Quota Calculator.

The Provisioned Throughput Quota Calculator automatically recommends the KTPM quota based on your workload. Fill in the following parameters, and the calculator will output the recommended input KTPM and output KTPM.

Parameter	Description	Impact on results
Requests per minute (RPM)	Number of requests per minute during peak business hours.	The larger the RPM, the more the recommended input and output KTPM increases proportionally.
Average input length (tokens)	Average number of input tokens per request.	The longer the input, the higher the tier and coefficient, and the higher the recommended input KTPM. Tier boundaries vary by model; refer to the console for actual values.
Average output length (tokens)	Average number of output tokens per request.	The longer the output, the higher the possible coefficient, and the higher the recommended output KTPM.
Cache hit rate (%)	The percentage of request prefix tokens that hit the cache. Actual hit rate depends on request content repetition; refer to actual runtime results.	A higher hit rate slows input quota consumption, reducing the recommended input KTPM. Affects input KTPM only, not output KTPM.

API response field descriptions

API responses for PTU deployments include the following quota-related fields that identify the billing method and quota consumption:

Field	Type	Description
`service_tier`	String	A top-level response field (consistent across all API formats). A value of `ptu-standard` indicates PTU quota is used. A value of `default` or no return indicates pay-as-you-go billing.
`provisioned_tokens`	Integer	The actual PTU quota tokens consumed after conversion (includes tiered coefficients and cache discounts).
`cached_tokens`	Integer	The number of tokens that hit the prefix cache. For details, see Context Cache.

The JSON paths for these fields differ by API format:

OpenAI Chat compatible
OpenAI Responses
Anthropic compatible
DashScope

Field	JSON path	Description
`cached_tokens`	`usage.prompt_tokens_details.cached_tokens`	Input-side cached token count
`provisioned_tokens`	`usage.prompt_tokens_details.provisioned_tokens`	Input-side PTU quota consumption
`provisioned_tokens`	`usage.completion_tokens_details.provisioned_tokens`	Output-side PTU quota consumption

Field	JSON path	Description
`cached_tokens`	`usage.input_tokens_details.cached_tokens`	Input-side cached token count
`provisioned_tokens`	`usage.input_tokens_details.provisioned_tokens`	Input-side PTU quota consumption
`provisioned_tokens`	`usage.output_tokens_details.provisioned_tokens`	Output-side PTU quota consumption

Field	JSON path	Description
`provisioned_tokens`	`usage.prompt_tokens_details.provisioned_tokens`	Input-side PTU quota consumption
`provisioned_tokens`	`usage.output_tokens_details.provisioned_tokens`	Output-side PTU quota consumption

The Anthropic-compatible format does not currently return the cached_tokens field. You can use provisioned_tokens to indirectly evaluate cache effectiveness.

Field	JSON path	Description
`cached_tokens`	`usage.prompt_tokens_details.cached_tokens`	Input-side cached token count
`provisioned_tokens`	`usage.prompt_tokens_details.provisioned_tokens`	Input-side PTU quota consumption
`provisioned_tokens`	`usage.completion_tokens_details.provisioned_tokens`	Output-side PTU quota consumption

For complete field definitions and value ranges, see API reference.

Monitoring and verification

You can monitor PTU deployments through the model monitoring feature on the Model Studio platform. The following metrics are relevant to long-input and caching:

PTU utilization: Three independent curves for input, output, and thinking mode output. In long-input scenarios, tiered coefficients can cause utilization to exceed 100%. This is expected behavior.
Token usage and cache hits: Includes the cached_tokens data series to show the proportion of cache hits relative to total input.
In-quota and out-of-quota call counts: Shows the proportion of requests that fall back to pay-as-you-go billing after exceeding PTU quota.

For more monitoring metrics and instructions, see Model monitoring.

FAQ

Q: What happens when PTU quota is exceeded?

The request automatically falls back to pay-as-you-go billing. In the API response, the service_tier field is either absent or returns default, and the response header includes x-dashscope-ptu-overflow:true. Service is not interrupted.

Q: What happens when a single input exceeds the model limit?

Qwen-series models have a 128K token input limit, and DeepSeek-series models have a 64K token limit. Requests exceeding the limit also automatically fall back to pay-as-you-go billing.

Q: How do I verify that caching is working?

Check the cached_tokens field in the API response. A value greater than 0 indicates a prefix cache hit. Cache-hit tokens consume quota at the model's discount coefficient (for specific discount rates, see Quota consumption rules). You can also view trends in the token usage chart on the console monitoring page.

Q: What if cached_tokens is always 0 and caching is not working?

Common causes: The input prefix is inconsistent between requests (for example, the System Message changes), the interval between two requests exceeds the cache validity period, or the input token count is too low to trigger caching. For troubleshooting methods and cache limits, see Context Cache.

Q: Why does utilization exceed 100%?

Some models (such as glm-5.1) have long-input tiered coefficients that cause actual quota consumption to exceed the raw token count. Utilization = converted consumption ÷ purchased quota. Exceeding 100% means the consumption rate exceeds the purchased quota. The excess automatically falls back to pay-as-you-go billing without affecting service availability.

​Overview

​Quota consumption rules

​Calculation examples (glm-5.1)

​Estimating quota with the Provisioned Throughput Quota Calculator

​API response field descriptions

​Monitoring and verification

​FAQ

Overview

Quota consumption rules

Calculation examples (glm-5.1)

Estimating quota with the Provisioned Throughput Quota Calculator

API response field descriptions

Monitoring and verification

FAQ