Skip to main content

Explicit cache (Qwen4)

Explicit cache for Qwen4 series models only. Use automatic caching or cache breakpoints to specify the input prefix to reuse, and set the cache time-to-live to 5 minutes or 1 hour to reduce repeated computation, latency, and input costs.

In scenarios such as multi-turn conversations, long-document Q&A, and agents, consecutive requests often share a large amount of identical leading content, such as system prompts, tool descriptions, or reference documents. With explicit cache (Qwen4), subsequent requests within the validity period read this content directly from the cache instead of recomputing it, which reduces response latency and input costs. Explicit cache uses cache breakpoints to specify the content prefix to reuse, and hits are deterministic: as long as the prefix is identical and within the validity period, subsequent requests read the cache. The implicit cache, which automatically identifies common prefixes, does not guarantee hits. Qwen4 adds automatic caching and cache TTL on top of basic usage. For details, see Advanced usage.
This page describes the Qwen4 version of the existing explicit cache: the content block breakpoint usage stays the same, and automatic caching and cache TTL are added. Its relation to the two cache modes in Context Cache:
  • Explicit cache: use cache breakpoints to specify the input prefix, with deterministic hits. This is what this page describes.
  • Implicit cache: when no cache parameters are set, common prefixes are identified automatically, and hits are not guaranteed. For details, see Implicit cache.
  • The cache_control content block marker of the earlier OpenAI-compatible Chat Completions API remains compatible and works the same way as prompt_cache_breakpoint. For details, see Basic usage.

How it works

Caches are created and read at breakpoints. With automatic caching, the system places the breakpoint at the end of the input. With cache breakpoints, you mark them on content blocks. The cached prefix for each breakpoint is all input from the start of the request to the end of the content block that contains the breakpoint. When it receives a request, the system uses each breakpoint as an end point, looks back up to 20 content blocks, and selects the longest identical prefix that has been cached:
  • Cache hit: The cached prefix is read and renewed based on the TTL of the current request.
  • Cache miss: The content from the start of the request to the breakpoint is cached as a new entry. If part of the prefix already hits the cache, only the new part is cached.
A cache hit requires an exactly identical prefix, and caches are reused only across requests from the same user. To improve the hit rate, place content that stays the same at the beginning and content that changes with each request after the breakpoint. If you set no cache parameters, the system uses the implicit cache to automatically identify common prefixes across requests, but hits are not guaranteed. To reliably reuse specific content, enable automatic caching or set cache breakpoints.

Basic usage

Set a cache breakpoint on the content block you want to reuse. The system uses the end of that content block as the breakpoint and treats all input from the start of the request to that point as the cached prefix: the first request creates the cache, and later requests with the same prefix within the validity period read it. The cached prefix must contain at least 1,024 tokens.
  • OpenAI compatible Chat Completions
  • OpenAI compatible Responses
  • Anthropic compatible
Add prompt_cache_breakpoint with the value {"mode": "explicit"} to the content block to cache in messages. The content of a message with a breakpoint must be an array:
messages=[
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": long_text,
        "prompt_cache_breakpoint": {"mode": "explicit"},  # <-- Cache breakpoint
      },
      {"type": "text", "text": "Summarize this document."},  # Content after the breakpoint is not part of this cache
    ],
  }
],
The existing "cache_control": {"type": "ephemeral"} content block marker remains compatible and works the same way as prompt_cache_breakpoint. For the same user, a cache for an identical prefix can be reused across the two markers. We recommend prompt_cache_breakpoint for new integrations.
Use only one of prompt_cache_breakpoint and cache_control in a request.
The following example sends two requests: the first creates the cache, and the second keeps the prefix unchanged and changes only the question to read the cache.
Python
import os
from openai import OpenAI

client = OpenAI(
  # If the environment variable is not set, replace the following line with: api_key="sk-xxx"
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  # This is the URL for the Singapore region. Replace WorkspaceId with the actual workspace ID. URLs vary by region.
  base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# A long document to reuse. The cached prefix must contain at least 1,024 tokens.
long_text = "<Fixed long document>" * 400

def ask(question):
  completion = client.chat.completions.create(
    model="qwen4-max",
    messages=[
      {
        "role": "user",
        "content": [
          {"type": "text", "text": long_text, "prompt_cache_breakpoint": {"mode": "explicit"}},  # <-- Cache breakpoint
          {"type": "text", "text": question},  # The content after the breakpoint can change each time
        ],
      }
    ],
    extra_body={
      "enable_thinking": False,
    },
  )
  return completion.usage.prompt_tokens_details

ask("Summarize this document.")  # First request: create the cache
details = ask("List three key points from this document.")  # Second request: read the cache
print(f"Cache read: {details.cached_tokens} tokens, cache write: {details.cache_write_tokens} tokens")
In the second request, cached_tokens is the number of tokens read from the cache, and cache_write_tokens is the number created by this request. When the prefix does not change, cache_write_tokens is 0. The numbers are illustrative, and the actual values depend on the document length.

Use multiple breakpoints

When a prompt consists of parts that change at different frequencies, you can set a breakpoint for each part. For example, a customer service prompt typically includes a system persona (rarely changes), knowledge base retrieval results (change with the inquiry topic), and conversation history (keeps growing). If you set a breakpoint at the end of each part, the system persona can still hit the cache even when the knowledge base content changes. Each request keeps at most the last 4 valid breakpoints. Additional breakpoints do not take effect. When automatic caching is enabled, the automatic breakpoint uses one of the 4 slots.

Renewal when a breakpoint moves

The system creates or renews only the breakpoints in the current request. After a breakpoint moves, the validity period of the earlier breakpoint is not extended, even if its cache is read. For example, with the same TTL, the breakpoint in the previous request is at token 2,048, and the current request moves the breakpoint to token 3,072 without keeping the earlier one. If the earlier breakpoint is within the lookback range and has not expired, the current request reads the first 2,048 tokens from the cache and caches the next 1,024 tokens. The system creates or renews only the breakpoint at token 3,072. The cache at token 2,048 is not renewed.

Advanced usage

Qwen4 adds the following two capabilities on top of basic usage. You can combine them:
  • Automatic caching: the system automatically places a breakpoint at the end of the input, which suits multi-turn conversations.
  • Cache TTL: choose 5 minutes or 1 hour.

Automatic caching

With automatic caching, the system automatically places a breakpoint at the end of the input of each request, so you do not need to move the breakpoint as the conversation grows. The automatic breakpoint does not include the template suffix used to start generating the response. Automatic caching can be used together with the cache breakpoints described in Basic usage. If a manual breakpoint already exists at the end of the input, no duplicate breakpoint is placed.
  • OpenAI compatible Chat Completions
  • OpenAI compatible Responses
  • Anthropic compatible
Set prompt_cache_options at the top level of the request and set mode to implicit. With the OpenAI Python SDK, pass it through extra_body:
completion = client.chat.completions.create(
  model="qwen4-max",
  messages=[{"role": "user", "content": "Summarize this document: " + long_text}],
  extra_body={
    "enable_thinking": False,
    "prompt_cache_options": {"mode": "implicit"},  # <-- Enables automatic caching
  },
)
The values of prompt_cache_options.mode and the corresponding behaviors are as follows:
SettingAutomatic breakpointCaching behavior
mode is implicitPlacedThe automatic breakpoint at the end of the input and manual breakpoints both take effect
mode is explicit, with manual breakpointsNot placedCreates or reads the cache at manual breakpoints
mode is explicit, without manual breakpointsNot placedUses the implicit cache
prompt_cache_options omitted, null, or {}, or only ttl setNot placedCaches at manual breakpoints if any; otherwise uses the implicit cache
When mode is implicit, the automatic breakpoint cache is used and follows the TTL and billing rules on this page. It is different from the implicit cache used when no cache parameters are set. Setting mode to explicit only means that no automatic breakpoint is placed. It does not disable the implicit cache. mode applies only to the current request.
The following complete example enables automatic caching and sends the same long-document request twice: the first request creates the cache, and the second reads it.
  • OpenAI compatible Chat Completions
  • OpenAI compatible Responses
  • Anthropic compatible
import os
from openai import OpenAI

client = OpenAI(
  # If you have not configured the environment variable, replace the following line with: api_key="sk-xxx"
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  # The following URL is for the Singapore region. Replace WorkspaceId with your actual workspace ID. URLs differ by region.
  base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# A long document to reuse. The cached prefix must contain at least 1,024 tokens.
long_text = "<Fixed long document>" * 400

def ask():
  completion = client.chat.completions.create(
    model="qwen4-max",
    messages=[{"role": "user", "content": "Summarize this document: " + long_text}],
    extra_body={
      "enable_thinking": False,
      "prompt_cache_options": {"mode": "implicit"},  # <-- Enables automatic caching
    },
  )
  return completion.usage.prompt_tokens_details

for i in range(1, 3):
  details = ask()
  print(f"Request {i}: cache read {details.cached_tokens} tokens, cache write {details.cache_write_tokens} tokens")
The output is similar to the following. The actual token counts depend on the document length:
Request 1: cache read 0 tokens, cache write 2048 tokens
Request 2: cache read 2048 tokens, cache write 0 tokens
The input of the second request is identical to that of the first request, so it reads the cache created by the first request.

Set the cache TTL

The cache time-to-live (TTL) can be 5m (5 minutes) or 1h (1 hour). The default is 5m. When the cache is hit within the validity period, it is automatically renewed. For details, see Renewal rules.
  • OpenAI compatible Chat Completions
  • OpenAI compatible Responses
  • Anthropic compatible
Set it in prompt_cache_options.ttl at the top level of the request. It applies to all breakpoints in the request:
extra_body={
  "prompt_cache_options": {
    "mode": "implicit",
    "ttl": "1h",  # <-- Cache TTL is 1 hour
  },
},
If prompt_cache_options.ttl is not set, the ttl of the last cache_control marker in the request is used. If that marker has no ttl, or the request has no cache_control marker, the TTL is 5m.TTL applies only to explicit cache. When you set ttl, also set a cache breakpoint or enable automatic caching.

Renewal rules

For the first request, or after the cache expires, the cache is created with the TTL of the current request. When a request hits a valid cache, renewal and billing follow these rules:
  • The later expiration time is kept, along with the higher tier. The new expiration time is approximately the later of the original expiration time and the start of cache processing for the current request plus the current TTL. For example, a request at 12:03 that hits a 5m cache expiring at 12:05 extends the expiration to about 12:08. As another example, when a 5m request hits a valid 1h cache, the renewal is calculated based on 5m, but the cache keeps its 1h tier, and subsequent 1h requests within the validity period are still billed as cache reads.
  • When upgrading from 5m to 1h, the upgraded portion is billed as 1h cache creation. When a 1h request hits a valid 5m cache, the hit portion is created again with a 1h TTL and is no longer billed as a read. If part of the prefix already has a valid 1h cache, that part is still billed as a read, and only the remaining portion and the new prefix are billed as creation. In all other cases (a 5m request hits a 5m cache, a 5m request hits a 1h cache, or a 1h request hits a 1h cache), the hit portion is billed as a read.
The validity period starts when cache processing for the current request begins. It is not added to the original expiration time and is not reset when the response ends. Response generation time counts toward the validity period, and the actual expiration time reserves a small amount of processing time. The following example shows consecutive requests for the same cache, which was created with a 1h TTL at 12:00 and originally expires at 13:00:
RequestTTL of this requestExpiration before the requestExpiration after the request (approx.)Billing and tier
Sent at 12:205m13:0013:00Cache read. 12:25 is earlier than the original expiration, so the expiration stays the same. It is neither shortened nor extended to 13:20.
Sent at 12:585m13:0013:03Cache read. The expiration is extended to 5 minutes after this request starts, and the cache keeps its 1h tier.
Sent at 13:011h13:0314:01Cache read. The cache is already in the 1h tier, so no upgrade creation is incurred.
The example above assumes serial requests with the same prefix. When multiple requests write to the same cache concurrently, they may overwrite each other, and the latest expiration time is not guaranteed to be kept.

View cache usage

The usage in the response returns the number of tokens read from and written to the cache. For streaming output with the OpenAI compatible Chat Completions API, set stream_options to {"include_usage": true} and read the usage from the last chunk that contains usage.
  • OpenAI compatible Chat Completions
  • OpenAI compatible Responses
  • Anthropic compatible
The following is the usage when the cache is created for the first time (text-only input, non-thinking mode):
{
  "usage": {
    "prompt_tokens": 2304,
    "completion_tokens": 128,
    "total_tokens": 2432,
    "prompt_tokens_details": {
      "text_tokens": 2304,
      "cached_tokens": 0,
      "cache_write_tokens": 2048,
      "cache_creation_input_tokens": 2048,
      "cache_type": "ephemeral",
      "cache_creation": {
        "ephemeral_5m_input_tokens": 2048,
        "ephemeral_1h_input_tokens": 0
      }
    },
    "completion_tokens_details": {}
  }
}
FieldDescription
prompt_tokensTotal number of input tokens, including cache reads, cache writes, and input that does not use the cache. It is not reduced by cache hits.
total_tokensSum of prompt_tokens and completion_tokens
prompt_tokens_details.cached_tokensNumber of tokens read from the cache
prompt_tokens_details.cache_write_tokensNumber of tokens written to the cache
prompt_tokens_details.cache_creation_input_tokensNumber of tokens written to the cache. Equals cache_write_tokens. Do not add the two together.
prompt_tokens_details.cache_typeType of the explicit cache. Always ephemeral.
prompt_tokens_details.cache_creationNumber of tokens written to the cache by TTL. The sum of ephemeral_5m_input_tokens and ephemeral_1h_input_tokens equals the number of tokens written to the cache.
In the example above, the input that does not use the cache is 2304 − 0 − 2048 = 256 tokens. All cache fields are raw token counts and do not include billing multipliers. When you send a request with the same prefix again within the validity period, cached_tokens is 2048, cache_write_tokens, cache_creation_input_tokens, and both items in cache_creation are 0, and prompt_tokens is still 2304.Input details for images, audio, and video are returned based on the actual input. Output details such as thinking are returned based on the model and configuration. Fields that do not apply are omitted rather than returned as 0.

Best practices

  • Use automatic caching for multi-turn conversations. The automatic breakpoint moves forward as the conversation grows, so you do not need to adjust breakpoints in each turn.
  • Put fixed content first. Place content that does not change, such as system prompts, tool definitions, and reference documents, at the beginning of the input. Place content that changes in each request, such as timestamps and user questions, at the end.
  • Place the breakpoint on the last content block that is identical across requests. Do not place a breakpoint on a content block that changes in each request. Otherwise, each request creates a new cache and never reads one.
  • Append to the history instead of rewriting it. Modifying, removing, or reordering earlier messages or tool definitions prevents later caches from being hit.
  • Use multiple breakpoints for content that changes at different frequencies. For example, if tool definitions rarely change but reference documents are updated daily, set a breakpoint at the end of each.
  • Add breakpoints when a turn adds many content blocks. Each breakpoint looks back up to 20 content blocks. If a turn adds more content than this range, add a breakpoint in the middle.
  • Choose the TTL based on the reuse interval. Use 5m when content is reused within 5 minutes, because each hit renews the cache. Use 1h when the reuse interval is between 5 minutes and 1 hour.
  • Monitor hits with the usage fields. The cache hit rate equals the number of tokens read from the cache divided by the total number of input tokens. If the hit rate is low, check whether the prefix and breakpoint positions are stable.

Billing

Cache breakpoints affect only how input tokens are billed. Output tokens are billed at the standard price.
Input token typeBilling
Cache write, TTL 5mApproximately 125% of the input token price
Cache write, TTL 1hApproximately 200% of the input token price
Cache readQwen4-Max
Neither read from nor written to the cacheInput token price
If the cached prefix of the current request contains an existing valid cache, the existing part is billed as a cache read, and only the new part is billed as a cache write. For billing when the TTL is upgraded from 5m to 1h, see Renewal rules.

Limits

  • Only Qwen4 series models are supported.
  • The cached prefix must contain at least 1,024 tokens. Shorter prefixes do not create a cache; the request succeeds, and cache write details grouped by TTL are omitted.
  • Each request keeps at most the last 4 valid breakpoints. When automatic caching is enabled, the automatic breakpoint uses one of them.
  • Using each breakpoint as an end point, the system looks back up to 20 content blocks. If the cache to hit is more than 20 content blocks away from the breakpoint, it cannot be hit.
  • TTL can only be 5m or 1h.

FAQ

Q: Can a request hit the cache without any cache parameters?

Yes. Without cache parameters, the implicit cache is used. The system automatically identifies common prefixes across requests, but hits are not guaranteed. To hit the cache reliably, set cache breakpoints or enable automatic caching.

Q: Why does the cache validity period not change when I set only the TTL?

TTL applies only to explicit cache. If you set only prompt_cache_options.ttl in the OpenAI compatible Chat Completions API, the request uses the implicit cache. Add a manual breakpoint as well, or set prompt_cache_options.mode to implicit.

Q: Does a 5m request that hits a 1h cache shorten the cache validity period?

No. The new expiration time is the later of the original expiration time and the renewal time of the current request, and the cache keeps its 1h tier. For details, see Renewal rules.