How to Diagnose OpenAI Prompt Cache Misses in the Responses API: Baselines, Reasons, Fixes, and Cost Checks

What changed on September 8: Prompt Cache Diagnostics is now generally available

OpenAI’s September 8, 2026 API changelog states that Prompt Cache Diagnostics is now generally available in the Responses API for GPT-5.6 and later supported models. The feature is designed for a narrow but operationally important task: compare cache reuse against a previous response, classify why a prompt-cache miss appears to have happened, and return troubleshooting guidance that helps teams preserve cacheable request structure. This tutorial treats the feature as a diagnostic instrument, not as an automatic cache-repair mechanism and not as a guarantee that future requests will reuse cached tokens.

The supported scope matters because prompt-caching bugs are easy to misdiagnose when teams mix API families or model generations. According to OpenAI’s diagnostics guide, Prompt Cache Diagnostics applies to Responses API requests using GPT-5.6 and later supported models. It does not apply to GPT-5.5, and it is not documented as Chat Completions functionality. If a migration plan still has GPT-5.5 workloads, Chat Completions clients, or wrappers that abstract away request shape, those workloads should be excluded from this diagnostic workflow until they are actually using a supported Responses API path.

The selected article explains GPT-6 Astra prompt caching concepts including explicit breakpoints, cache keys, a 30-minute TTL, and long-context cost control. The complete GPT-6 Astra Prompt Caching Guide: Explicit Breakpoints, Cache Keys, 30-Minute TTL, and Long-Context Cost Control article provides the destination-specific detail for this section’s OpenAI Prompt Caching Guide decision because it is the closest match for a deeper OpenAI prompt caching reference and directly supports discussion of cache keys, TTL behavior, and long-context cache economics.

The practical value of general availability is that teams can stop guessing whether a miss was caused by a model switch, a tool-array change, an output-format change, a service-tier change, conversation compaction, or an input-prefix mutation. Before diagnostics, many teams could only compare ordinary usage counters and request logs, then infer what changed. With diagnostics, the API can return a classified result tied to a comparison response, while ordinary usage fields still remain the source of truth for how many input tokens were actually served from cache.

The core idea: compare a new request with a recent completed baseline

Prompt Cache Diagnostics works by adding a diagnostic baseline to a Responses API request. OpenAI documents this through `prompt_cache_options`, where you set `comparison_response_id` to a recent completed response from the same organization. That response is not used as conversation memory merely because it is named in the diagnostic option; it is used as the comparison point for analyzing cache reuse.

{
  "model": "<GPT-5.6-or-later-supported-model>",
  "input": [
    {
      "role": "user",
      "content": "<current request content>"
    }
  ],
  "prompt_cache_options": {
    "comparison_response_id": "<recent_completed_response_id_from_same_org>"
  }
}

The comparison response must be recent, completed, and from the same organization. OpenAI also notes that diagnostic records expire after a short period, so a response that was a valid baseline earlier may no longer be available for diagnostics later. In production observability terms, you should treat the baseline as short-lived evidence: capture it during a controlled repro window, run one change at a time, and store your own request metadata in application logs rather than assuming OpenAI’s diagnostic record will remain queryable indefinitely.

A common failure mode is confusing `comparison_response_id` with `previous_response_id`. OpenAI’s diagnostics guide separates these controls. `previous_response_id` carries conversation context for continuation. `comparison_response_id` selects the diagnostic baseline and does not load the previous response’s content into the new request. The two IDs can be the same or different, but they answer different questions: one controls conversational state, and the other controls diagnostic comparison.

Why cache reuse can drift even when the prompt “looks the same”

Prompt-cache reuse depends on stable request structure, not on a human impression that two prompts are similar. A developer may keep the same visible system instruction while changing the model, service tier, tool definitions, tool order, output schema, reasoning effort, verbosity, or early input prefix. Any of those changes can make a request less reusable or non-reusable against the prior cached prefix, even if the user-facing task is unchanged.

OpenAI documents several classified miss reasons that illustrate this drift: `model_changed`, `prompt_cache_key_changed`, `service_tier_changed`, `tools_changed`, `text_format_changed`, `reasoning_effort_changed`, `verbosity_changed`, `context_compacted`, and `input_changed`. These categories are useful because production systems often introduce small changes outside the prompt author’s view. A routing layer may change the service tier, a tool registry may reorder tools after deployment, a schema generator may emit fields in a different order, or a conversation manager may compact old context to keep the request within operational limits.

The highest-risk drift usually happens near the beginning of the input because prompt caching rewards stable prefixes. If your application prepends dynamic request IDs, timestamps, user-specific debug labels, randomized policy snippets, or non-deterministically ordered tool declarations before the stable instruction block, you may reduce reuse even though the semantic instruction set remains similar. A better pattern is to keep the cacheable prefix stable and move volatile details later when the application design allows that without changing task correctness.

Cache drift can also be intentional. OpenAI’s diagnostics guide notes that `context_compacted` may be reported when compaction changes reuse, but compaction can still lower total cost if it reduces enough input tokens. That distinction is important for cost reviews: the goal is not always to maximize cached tokens; the goal is to minimize total cost and latency for the workload while preserving output quality, policy requirements, and operational reliability.

Diagnostic evidence is not the same as ordinary caching metrics

Ordinary prompt-caching metrics answer a quantitative billing and usage question: how many input tokens were cached for this request? In the Responses API, OpenAI identifies actual reuse through `usage.input_tokens_details.cached_tokens`. If that number rises, the request reused more cached input tokens; if it falls, the request reused fewer. This field is the cost-accounting evidence you should use when calculating whether a fix improved real cache reuse.

Prompt Cache Diagnostics answers a different question: when comparing this request to a selected baseline, what is the first classified reason OpenAI can identify for a cache miss or comparison failure? OpenAI’s documentation states that diagnostics report the first classified reason, not every difference. That means a diagnostic result can be directionally useful while still incomplete. If the API reports `tools_changed`, you should not assume the model, service tier, schema, reasoning effort, verbosity, and input prefix are all identical; you should fix the reported issue, rerun a controlled request, and inspect the next result and usage counters.

Recommendation: treat diagnostics as a triage tool and `cached_tokens` as the measurement tool. A diagnostic `cache_hit` can tell you that no miss was detected against the comparison, but it does not prove that every input token was cached.

This distinction prevents two expensive mistakes. First, a team might see `cache_hit` and wrongly assume full-token reuse, then overstate savings in a finance review. Second, a team might see a low cached-token count and assume diagnostics failed, when the actual request structure was only partially cacheable or the compared prefix was shorter than expected. A reliable workflow records both the diagnostic result and the ordinary usage fields for every repro run.

The selected article is a production-focused masterclass on building scalable applications with the OpenAI Responses API, covering authentication, streaming optimization, function calling, and deployment concerns. The complete The Complete ChatGPT API Integration Masterclass: Building Production Applications with the Responses API in 2026 article provides the destination-specific detail for this section’s Responses API Production Tutorial decision because it matches the marker’s need for a broader Responses API production tutorial that complements a cache-miss diagnostic workflow.

The four diagnostic result types you need to recognize before troubleshooting

OpenAI documents four result types for Prompt Cache Diagnostics: `cache_hit`, `cache_miss`, `comparison_response_not_found`, and `unavailable`. These are not interchangeable, and they should lead to different next actions in a production debugging session.

The `cache_hit` result is the easiest to overread. OpenAI explicitly distinguishes a diagnostic hit from full-token reuse: a hit means no miss was detected against the comparison, not that every input token was cached. For example, a request could share the stable prefix with the baseline and still include new tail content that is not cached. The correct cost check is to compare total input tokens, cached input tokens, and the effective cached share for the request family you are testing.

The `cache_miss` result is actionable but not exhaustive. Because diagnostics report the first classified reason, a single miss reason should become the next hypothesis, not the entire root-cause report. If the reason is `tools_changed`, freeze the tool list and ordering before investigating schema format or verbosity. If the reason is `input_changed`, compare the earliest portion of the assembled request, not only the visible user message, because system and developer instructions often live outside the UI that prompted the bug report.

The `comparison_response_not_found` result usually points to baseline hygiene rather than model behavior. The baseline may be too old for short-lived diagnostic records, may not be completed, may come from a different organization, or may have been logged incorrectly by the application. A robust debugging run captures the response ID immediately after a completed baseline request and uses it promptly for a small series of one-variable comparisons.

The `unavailable` result is a reminder that diagnostics are best effort. OpenAI states that diagnostics never block or fail the request and do not change model output. In other words, generation can succeed while the diagnostic record is unavailable. Production monitors should therefore avoid alerting as if `unavailable` were an application outage; instead, they should mark the diagnostic evidence as missing and rely on normal request status, latency, usage, and billing telemetry.

How this tutorial will use diagnostics without overstating them

The workflow in this tutorial starts with a clean baseline, then tests one potential drift source at a time: model, cache key, service tier, tools, text format, reasoning effort, verbosity, compaction, and input changes. For each step, the decision rule is the same: change only one variable, run a supported Responses API request with `comparison_response_id`, inspect the diagnostic result, and verify actual reuse with `usage.input_tokens_details.cached_tokens`.

Diagnostics add no separate cost or rate-limit charge according to OpenAI’s guide, but the baseline request and every retry request are still normal API requests that are billed normally. That means a cost-conscious team should not run broad diagnostic sweeps against production-scale payloads until it has a small, representative repro case. The efficient sequence is to preserve the exact assembled request metadata, reproduce with a recent completed baseline, fix the first classified reason, and only then validate against a larger workload sample.

OpenAI also states that Prompt Cache Diagnostics is compatible with Zero Data Retention because it does not store raw prompts or model outputs for diagnostics; records contain organization-scoped configuration metadata, token estimates, and hashes, and they expire after a short period. Enterprise administrators should still document this behavior in their internal data-handling review, because the diagnostic workflow depends on short-lived metadata and response identifiers even though OpenAI says raw prompts and outputs are not stored for diagnostics.

The opening principle is simple: use diagnostics to explain drift, use usage fields to measure reuse, and use cost checks to decide whether a fix is worth keeping. A perfect-looking cache pattern is not the objective if conversation compaction, routing discipline, or shorter prompts reduce total spend while preserving the required output. The rest of the tutorial turns that principle into a repeatable debugging runbook for advanced Responses API teams.

Build a clean diagnostic baseline before you chase cache misses

A useful prompt-cache investigation starts with a baseline response that is recent, completed, and produced by the same OpenAI organization as the request you want to diagnose. OpenAI’s Prompt Cache Diagnostics compare a new Responses API request against a previous response ID supplied as comparison_response_id inside prompt_cache_options; that ID is diagnostic evidence, not conversation memory, and it does not force the platform to reuse a particular cache entry.

The safest baseline is a request you can reproduce from a user-owned, versioned prompt asset such as a policy file, contract template, product catalog excerpt, or internal operating procedure. In the example below, the policy file is a local file controlled by the API user, and the diagnostic run never depends on an external website, changing documentation page, or generated prompt text that could mutate between calls.

This section uses a stepwise Python pattern that separates four concerns: the stable prompt prefix, the stable prompt_cache_key, the conversation parent carried by previous_response_id, and the diagnostic comparison baseline carried by prompt_cache_options.comparison_response_id. Keeping those roles separate prevents a common production mistake: treating the comparison response as if it reloads prior content or continues the conversation.

Step 1: Prepare a user-owned policy file and a deterministic prompt prefix

Create a local policy file that your organization owns or is authorized to process, then keep its contents stable while testing cache behavior. If you edit the first portion of the prompt between baseline and comparison requests, diagnostics can legitimately classify the request as input_changed, and that is not an API failure.

from pathlib import Path
import hashlib
import json
import os
import time

from openai import OpenAI

client = OpenAI()

# Use a GPT-5.6 or later supported model for Prompt Cache Diagnostics.
# Keep this value stable across the baseline and diagnostic request.
MODEL = os.environ["OPENAI_RESPONSES_MODEL"]

# Keep the key stable for the asset and diagnostic run.
# In production, choose a value that represents the stable prompt family/version.
PROMPT_CACHE_KEY = os.environ.get(
    "PROMPT_CACHE_KEY",
    "support-policy-v1-diagnostic-run"
)

# Use a user-owned or authorized local file. Do not fetch changing web content
# during cache diagnostics, because the prompt prefix must remain stable.
POLICY_PATH = Path(os.environ.get("POLICY_PATH", "policy/support-policy.md"))

def read_policy_file(path: Path) -> str:
    if not path.exists():
        raise FileNotFoundError(
            f"Policy file not found: {path}. Create a user-owned file before testing."
        )
    text = path.read_text(encoding="utf-8")
    if not text.strip():
        raise ValueError("Policy file is empty; use a stable, substantive policy document.")
    return text

def sha256_text(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()

policy_text = read_policy_file(POLICY_PATH)
policy_hash = sha256_text(policy_text)

stable_prefix = f"""You are assisting an internal support operations team.

Use the policy document below as the controlling source for this task.
Do not invent policy terms that are not present in the document.

<policy_document sha256="{policy_hash}">
{policy_text}
</policy_document>
"""

baseline_input = stable_prefix + """

Task:
Summarize the three most important obligations a support agent must follow.
Return a concise numbered list.
"""

The sha256 value in this example is a local integrity check for your own logs and test notes; it is not a cache key, a security boundary, or evidence that OpenAI stores the policy text for diagnostics. OpenAI states that Prompt Cache Diagnostics are compatible with Zero Data Retention because diagnostic records do not store raw prompts or model outputs; records use organization-scoped configuration metadata, token estimates, and hashes, and expire after a short period.

For test isolation, avoid running this first against a live production prompt that changes during the same hour. Copy the policy into a diagnostic fixture, freeze the fixture for the run, and change only one variable at a time: model, service tier, tools, text format, reasoning effort, verbosity, prompt key, or prompt input. If multiple fields change together, OpenAI notes that diagnostics report the first classified reason rather than every difference, so a noisy test can hide the real operational drift.

Step 2: Create and store a recent completed baseline response

The baseline request is billed like any other Responses API request, even though the diagnostic comparison you perform later has no separate diagnostic charge. Treat the baseline as a real API call: store the response ID only if the request completes, retain the minimum metadata needed for troubleshooting, and do not log the raw policy text or generated answer unless your organization’s data-handling policy permits it.

def to_plain_dict(obj):
    """Serialize SDK response objects without assuming a specific Python class."""
    if hasattr(obj, "model_dump"):
        return obj.model_dump()
    if hasattr(obj, "to_dict"):
        return obj.to_dict()
    if isinstance(obj, dict):
        return obj
    return json.loads(json.dumps(obj, default=str))

def extract_usage_summary(response_obj):
    data = to_plain_dict(response_obj)
    usage = data.get("usage") or {}
    input_details = usage.get("input_tokens_details") or {}
    return {
        "input_tokens": usage.get("input_tokens"),
        "output_tokens": usage.get("output_tokens"),
        "total_tokens": usage.get("total_tokens"),
        "cached_tokens": input_details.get("cached_tokens"),
    }

def safe_response_log(label, response_obj, extra=None):
    data = to_plain_dict(response_obj)
    record = {
        "label": label,
        "response_id": data.get("id"),
        "status": data.get("status"),
        "model": data.get("model"),
        "prompt_cache_key": PROMPT_CACHE_KEY,
        "policy_sha256": policy_hash,
        "usage": extract_usage_summary(data),
        "created_at_local_epoch": int(time.time()),
    }
    if extra:
        record.update(extra)
    print(json.dumps(record, indent=2, sort_keys=True))

baseline_response = client.responses.create(
    model=MODEL,
    prompt_cache_key=PROMPT_CACHE_KEY,
    input=baseline_input,
)

baseline_data = to_plain_dict(baseline_response)
if baseline_data.get("status") not in (None, "completed"):
    raise RuntimeError(
        f"Baseline response did not complete; do not use it for diagnostics: "
        f"{baseline_data.get('status')}"
    )

baseline_response_id = baseline_data["id"]
safe_response_log("baseline_completed", baseline_response)

The code accepts a missing status field because SDK object shapes can vary, but it refuses to use a known non-completed response as a comparison baseline. The operational rule is stricter than the code: only use a recent completed response from the same organization, because OpenAI documents comparison_response_not_found when the comparison record cannot be used, including cases where the diagnostic record has expired.

The selected article covers preparing api.openai.com for IPv6 with dual-stack DNS, firewall and proxy checks, observability, and rollback planning. The complete How to Prepare api.openai.com for IPv6: Dual-Stack DNS, Firewalls, Proxies, Observability, and Rollback article provides the destination-specific detail for this section’s OpenAI API Observability decision because it is the most observability-oriented OpenAI API operations post in the candidate set and fits sections about diagnosing API behavior through operational signals.

Step 3: Use previous_response_id for conversation state, not for diagnostics

In the Responses API, previous_response_id carries conversation context forward. It is appropriate when the user is asking a follow-up question and you want the model to continue from an earlier response. It is not the same as comparison_response_id, which only selects the diagnostic baseline used to classify cache reuse against a previous response.

The two IDs can be the same in a simple test, but they do not have to be. A realistic production conversation might use the most recent user-facing response as previous_response_id while using a separate, recent baseline response as comparison_response_id to determine whether the stable prompt prefix is still reusable.

# For the first follow-up in this controlled example, the baseline also acts
# as the conversation parent. In a real multi-turn conversation, this could be
# the latest completed response in the user's thread instead.
conversation_parent_response_id = os.environ.get(
    "PREVIOUS_RESPONSE_ID",
    baseline_response_id
)

followup_input = """
Follow-up task:
Draft a short agent checklist for applying the policy during a refund escalation.
Keep the answer under 120 words.
"""

This separation matters when a cache issue appears during a live conversation. If you put the prior conversation ID only in previous_response_id and omit prompt_cache_options.comparison_response_id, you have continued the conversation but have not requested diagnostic comparison against a baseline. If you put a response ID only in comparison_response_id, you have requested diagnostics but have not loaded that response as conversation context.

Step 4: Send a streaming diagnostic request and read diagnostics from response.completed

OpenAI documents that, for streaming responses, diagnostics are read from event.response in the response.completed event. Do not attempt to finalize cache diagnostics from early text deltas, partial tool events, or local assumptions while the stream is still open.

def find_diagnostic_objects(value):
    """
    Recursively find likely prompt-cache diagnostic objects in the serialized
    completed response. This keeps logging resilient if your SDK wraps fields.
    """
    found = []

    if isinstance(value, dict):
        for key, child in value.items():
            key_lower = str(key).lower()
            if "diagnostic" in key_lower or "prompt_cache" in key_lower:
                found.append({key: child})
            found.extend(find_diagnostic_objects(child))
    elif isinstance(value, list):
        for child in value:
            found.extend(find_diagnostic_objects(child))

    return found

stream = client.responses.create(
    model=MODEL,
    prompt_cache_key=PROMPT_CACHE_KEY,
    previous_response_id=conversation_parent_response_id,
    prompt_cache_options={
        "comparison_response_id": baseline_response_id
    },
    input=followup_input,
    stream=True,
)

completed_response = None

for event in stream:
    event_data = to_plain_dict(event)
    if event_data.get("type") == "response.completed":
        # OpenAI documents that streaming diagnostics are available on
        # event.response in the response.completed event.
        completed_response = event_data.get("response")
        break

if completed_response is None:
    raise RuntimeError("The stream ended before a response.completed event was captured.")

diagnostic_candidates = find_diagnostic_objects(completed_response)

safe_response_log(
    "diagnostic_completed",
    completed_response,
    extra={
        "comparison_response_id": baseline_response_id,
        "previous_response_id": conversation_parent_response_id,
        "diagnostic_candidates": diagnostic_candidates,
    },
)

The recursive extraction helper is intentionally conservative: it logs only objects whose keys indicate prompt-cache diagnostics or diagnostic metadata, rather than dumping the full response body. In a production client, replace the helper with the exact field access pattern supported by your SDK version, but keep the same data-minimization principle.

A diagnostic cache_hit means OpenAI did not detect a cache miss against the comparison; it does not mean every input token was served from cache. The cost and reuse check still comes from usage.input_tokens_details.cached_tokens, so your log should keep both the diagnostic result and the ordinary usage details for the completed response.

Step 5: Log enough to debug without storing sensitive prompts

Safe logging for prompt-cache diagnostics should answer five questions without copying the user’s prompt or the model’s answer into operational logs: which response was used as the baseline, which response was generated, which model and prompt key were used, what local prompt asset hash was present, and how many cached input tokens were reported in usage.

{
  "recommended_log_fields": {
    "response_id": "ID of the diagnostic response",
    "comparison_response_id": "Recent completed baseline response ID",
    "previous_response_id": "Conversation parent response ID, if used",
    "model": "Configured GPT-5.6-or-later supported model",
    "prompt_cache_key": "Stable key used for both calls",
    "policy_sha256": "Local hash of the user-owned policy file",
    "diagnostic_result": "cache_hit, cache_miss, comparison_response_not_found, or unavailable",
    "diagnostic_reason": "First classified miss reason, when provided",
    "cached_tokens": "usage.input_tokens_details.cached_tokens",
    "input_tokens": "usage.input_tokens",
    "total_tokens": "usage.total_tokens"
  },
  "avoid_logging": [
    "raw policy text",
    "raw user messages",
    "raw model output",
    "secrets, credentials, access tokens, or personal data not needed for debugging"
  ]
}

The diagnostic result can be unavailable because OpenAI describes the feature as best effort; diagnostics never block or fail the request and do not change the model output. Your application should therefore treat diagnostics as troubleshooting evidence, not as an availability dependency or a gate that must succeed before showing an answer.

Step 6: Isolate tests so the first miss reason is actionable

When you run the diagnostic request, change only the field you intend to test. If the baseline used one model and the diagnostic request uses another, model_changed can be the first classified miss reason. If the baseline used one prompt key and the diagnostic request uses another, prompt_cache_key_changed can be the first classified reason. If both changed, the diagnostic output may not tell you which change mattered most for your operational fix.

The last case is important because a cache miss is not automatically a cost regression. OpenAI documents context_compacted as a miss reason, but intentional compaction can still reduce total cost when the request sends fewer input tokens overall. Diagnose reuse first, then compare total billed usage for the old and new request shapes before deciding whether to revert.

Classify the miss reason, then change only the thing that explains it

OpenAI’s Prompt Cache Diagnostics are most useful when you treat the reported reason as a hypothesis to test, not as a complete diff of the request. The diagnostic record reports the first classified miss reason it can identify against the selected comparison response, and OpenAI states that diagnostics are best effort and can return unavailable. Your operational response should therefore be: confirm the baseline is valid, isolate the reported variable, run one controlled request, and verify actual reuse with usage.input_tokens_details.cached_tokens rather than assuming that a diagnostic cache_hit means every eligible token was reused.

The table below maps the documented miss reasons to the request surface you should inspect first, the most likely engineering cause, and the safest fix. Use it as a triage sheet during incident review, cost regression analysis, or SDK migration testing.

Fix model_changed: make routing observable before you blame caching

A model change is the cleanest miss to debug because it usually comes from configuration rather than prompt text. Inspect the effective model sent to the Responses API, not the model value you expected from application defaults. In production systems, the model can be changed by environment-specific config, experiment bucketing, task classifiers, emergency fallbacks, or SDK wrappers that select a default when the caller omits a value.

The safest fix is to treat the model as part of the cache-sensitive contract for a workload. Store the intended model beside the baseline response ID, log the effective model for every diagnostic request, and prevent mixed-model comparisons inside the same experiment notebook or cost dashboard. If you are evaluating a newer supported model, create a fresh baseline for that route instead of trying to compare the new request with a response generated on the older route.

Fix prompt_cache_key_changed: use keys for isolation, not accidental fragmentation

A prompt cache key change means the request no longer matches the baseline’s cache-key grouping. This can happen when teams place volatile metadata in the key, such as a request ID, session ID, timestamp, deployment hash, or A/B test label. That pattern fragments reuse because two prompts with identical stable prefixes can be placed into separate key spaces simply because unrelated metadata changed.

Use the cache key to encode a real reuse boundary. Good candidates are stable prompt-family versions, tenant isolation requirements, or compliance domains where reuse must not cross a boundary. Poor candidates are values that change on every call. Document the key derivation rule in the same place as model and service-tier routing, because a cache-key adjustment can change cost behavior even when the visible prompt text stays identical.

The selected article presents an AI cost optimization playbook with tactical patterns for reducing production LLM spend without sacrificing quality. The complete AI Cost Optimization Playbook: Cut LLM Bills 80% article provides the destination-specific detail for this section’s LLM Cost Optimization decision because it directly supports the current article’s cost-checking angle by giving readers a broader framework for reducing LLM bills beyond prompt cache fixes.

Fix service_tier_changed: keep latency policy separate from prompt experiments

A service-tier miss indicates that the baseline and the diagnostic request were sent with different tier settings. This often appears during incident response when teams reroute urgent traffic, or during load testing when a harness uses one tier while the application uses another. The prompt may be byte-for-byte similar, but the request configuration is not the same comparison target.

The fix is not always to force one tier everywhere. Instead, define service-tier policy per workload and maintain separate baselines for materially different routes. A high-priority customer-support escalation flow may deserve a different tier than a background summarization job. What matters for diagnostics is that you do not compare those two routes and then misread a service-tier miss as a prompt-regression problem.

Fix tools_changed: stabilize both tool schemas and tool ordering

OpenAI lists tools_changed as a documented miss reason, and the practical trap is that “same tools” must include the request representation your application sends. If your tool array is assembled from a map, plugin registry, feature-flag service, or build-time code generator, the order and schema details can drift even when humans believe the available capabilities are unchanged.

Normalize tool construction before diagnostics. Keep a canonical tool order for each prompt family, version tool schemas when arguments or descriptions change, and avoid injecting tools that the current task does not need. If a retrieval tool, database function, or image-generation tool is required only for a later step, do not place it into the stable prefix for every request unless the workload actually benefits from that capability being present.

{
  "diagnostic_test_rule": "Change one tool variable at a time",
  "baseline": {
    "tools": ["lookup_policy", "create_ticket"],
    "tool_schema_version": "support-v3"
  },
  "test_a": {
    "change": "same tools, same schemas, deterministic order",
    "expected_debug_value": "rules out non-deterministic ordering"
  },
  "test_b": {
    "change": "add one new tool only",
    "expected_debug_value": "confirms whether the new capability explains tools_changed"
  }
}

Fix text_format_changed: version output contracts deliberately

A text-format miss usually points to a changed output contract. Examples include switching from prose to structured JSON, adding a required field to a schema, changing a response envelope, or moving from a terse classification output to a multi-section explanation. These changes are legitimate product changes, but they should not be mixed into a cache reuse investigation without a new baseline.

Treat text format as an API contract between the model and your downstream system. If a billing pipeline expects strict fields, a schema change should receive a new prompt-family version and a new diagnostic baseline. If a human-facing feature allows users to request “brief” or “detailed” responses, do not let that preference silently alter the same cached prompt route unless you have decided that lower reuse is acceptable for personalization.

Fix reasoning_effort_changed and verbosity_changed: separate quality profiles

Reasoning effort and verbosity are different controls, but they create a similar diagnostic problem: the user-facing prompt may look unchanged while the request profile has changed. A system that automatically raises reasoning effort for hard cases or lowers verbosity for mobile surfaces should not compare those requests against one generic baseline and expect stable diagnostics.

Create named request profiles such as “triage concise,” “analysis detailed,” and “audit high-effort” if those concepts exist in your product. Each profile should have its own model route, reasoning effort, verbosity setting, output format, and baseline response ID. This prevents a cost dashboard from showing an apparent cache regression when the real change was a deliberate quality or presentation decision.

Fix context_compacted: compare total cost, not just cache reuse

context_compacted is the miss reason most likely to be misinterpreted by teams managing long conversations. Compaction can rewrite or summarize earlier context, which may reduce reuse against an earlier baseline. But OpenAI’s diagnostics are not a command to preserve every old token forever. A compacted conversation can be cheaper overall if it sends far fewer input tokens, even when the cached-token count falls.

Use a three-part check before reversing a compaction strategy. First, compare total input tokens before and after compaction. Second, compare cached input tokens, not just the diagnostic reason. Third, evaluate whether the output still satisfies the task’s factual, formatting, and safety requirements. If compaction lowers total input substantially and quality remains acceptable, the right fix may be to create a new post-compaction baseline rather than disabling compaction.

The selected article discusses GPT-5.5 long-context pricing, cache economics, regional uplift, and session budgeting for very large context windows. The complete GPT-5.5 Beyond 272K Tokens: Long-Context Pricing, Cache Economics, Regional Uplift, and Session Budgeting article provides the destination-specific detail for this section’s Long Context Conversation Management decision because it fits because cache misses often become expensive in long-context workloads, and the target adds useful context on budgeting and cache economics for large sessions.

Fix input_changed: protect the early stable prefix

input_changed is the broadest reason and often requires the most disciplined prompt engineering. The most damaging form is an early change to content that should have been stable: inserting the current date above the policy block, placing a request ID in the first instruction, reordering a long rubric, or regenerating a tool description with slightly different wording. Because prompt caching depends on reuse of a matching prefix, early drift can reduce reuse more than an appended user-specific tail.

Use an append-only prefix discipline. Put stable system instructions, durable policies, reusable rubrics, and fixed tool explanations first. Append volatile material later: the current user question, retrieved passages, timestamps, session metadata, and experiment annotations. When the shared policy must change, version it explicitly and create a new baseline; do not let automated formatting tools rewrite the stable prefix on unrelated deploys.

Recommended prompt layout for cache-sensitive workloads:

1. Stable instruction block
   - product role
   - safety and policy constraints
   - durable formatting rules

2. Stable task rubric
   - evaluation criteria
   - domain definitions
   - examples that rarely change

3. Stable tool description block
   - deterministic order
   - versioned schemas

4. Volatile request tail
   - user question
   - retrieved context
   - current timestamp if required
   - request-specific metadata only if the model needs it

Run one-variable-at-a-time experiments before changing production prompts

A reliable diagnostic experiment changes exactly one request variable between the baseline-shaped request and the candidate request. If you change the model, cache key, tool list, output format, and prompt text in the same deployment, diagnostics may report only the first classified reason, leaving the remaining differences hidden until the first issue is fixed. That behavior is expected from the source documentation and should shape your test plan.

1. Confirm the baseline is usable. The comparison response should be recent, completed, and from the same organization. If diagnostics return comparison_response_not_found, refresh the baseline before investigating prompt content.
2. Freeze the request envelope. Keep model, service tier, cache key, tools, text format, reasoning effort, and verbosity identical to the baseline while you test prompt text.
3. Change only the suspected field. If the reason is tools_changed, alter only the tool list or ordering. If it is input_changed, alter only prompt content.
4. Send a billed test request with diagnostics enabled. OpenAI states diagnostics add no separate charge or rate-limit cost, but the request itself is still a normal billed request.
5. Record both diagnostic result and usage. Store the diagnostic result, miss reason if present, total input tokens, cached tokens, model, cache key, service tier, and a hash or version ID for the stable prefix.
6. Stop after the first confirmed cause. Apply the smallest fix, create a new baseline if the change is intentional, and only then test the next suspected variable.

Use append-only prefix reviews in code review and prompt review

For production systems, prompt cache reliability should be reviewed like an API compatibility concern. A harmless-looking edit to the top of a prompt can have a larger cost impact than a long user-specific appendix. Require reviewers to identify whether a change touches the stable prefix, the volatile tail, request configuration, tool schemas, or output format. This keeps cache behavior visible before a release reaches live traffic.

A practical review rule is: if the change modifies stable content before the first user-specific section, it needs a prompt-version note and a fresh baseline. If it only appends task-specific context after the stable prefix, it may preserve reuse, but you should still verify with cached-token usage. If it changes the model, tier, tools, format, reasoning effort, or verbosity, it is not a prompt-text-only change and should be tested as a request-profile change.

Operational warning: do not “fix” every cache miss by freezing the application. New models, new schemas, new tools, refreshed policies, user corrections, and context compaction can all be correct product decisions. The goal is to distinguish accidental drift from intentional change, then measure whether the resulting cost, latency, and output quality are acceptable.

Decide when a miss is the right engineering tradeoff

Some cache misses are the cost of correctness. A regulated workflow may need a new policy prefix immediately after a compliance update. A support agent may need a new tool to create tickets instead of merely drafting replies. A research assistant may need higher reasoning effort for a complex analysis task. A long-running conversation may need compaction because the full transcript is no longer the best input representation.

Use a decision record for intentional misses. Capture the old baseline ID, new baseline ID, reported miss reason, reason for accepting the change, expected effect on total input tokens, and acceptance criteria for output quality. This avoids repeated rediscovery of the same miss in future reviews and gives finance, platform, and product teams a shared explanation when cached-token percentages move after a deliberate release.

The final cost check should always use real usage fields from the Responses API response. Diagnostics tell you why reuse differed from the comparison; usage tells you how many input tokens were actually cached. A request with a diagnostic cache_hit can still have uncached input tokens, and a request with an intentional miss can still be the cheaper design if it sends fewer total tokens or avoids unnecessary retries, tools, or long-context baggage.

Turn diagnostics into measurement, not anecdotes

A prompt-cache investigation is only useful if it separates three questions that often get blurred together: whether the diagnostic comparison found a mismatch, whether the request actually reused cached input tokens, and whether the request cost less than the uncached baseline. OpenAI’s Prompt Cache Diagnostics can help classify a miss against a recent completed response, but OpenAI also states that diagnostics are best effort, do not change model output, and can return unavailable. Treat each diagnostic result as one measurement point in a controlled experiment, not as a permanent verdict on the prompt.

How to read the token fields without overclaiming cache reuse

The most important production metric remains usage.input_tokens_details.cached_tokens. OpenAI’s diagnostics guide states that a diagnostic cache_hit means no miss was detected against the comparison; it does not mean every input token was cached. The cached_tokens value in usage is the field you use to measure actual input-token reuse on the completed request. If it is zero or lower than expected, you still have a cost and routing problem to investigate even when the diagnostic result says cache_hit.

Use cache_write_tokens as a write-side measurement when it is present in diagnostic or usage-related output for your model and response shape. Operationally, it tells you that some input tokens were not served from an existing cache entry and were instead eligible to be written for possible future reuse. This is useful when evaluating a new baseline, because the first request in a family of similar prompts is normally the expensive request that establishes reusable material, while later requests should show higher reuse if the stable prefix, model, tools, service tier, and other cache-relevant settings remain aligned.

Use comparison_reusable_tokens as a diagnostic estimate of how many tokens from the comparison response’s configuration and input shape appear reusable for the new request. This number is not a billing credit and does not mean the system will necessarily reuse exactly that many tokens. It is a troubleshooting signal that helps you size the opportunity: a large reusable estimate with low actual cached_tokens points to a mismatch, expiry, unavailable diagnostic data, or a request shape that diverged earlier than reviewers expected.

Use cache_missed_tokens as a diagnostic estimate of the reuse opportunity that was not realized for the comparison being checked. This field is most useful when paired with the classified miss reason. For example, a tools_changed reason with many missed tokens usually means the team should inspect tool definitions and ordering before rewriting the natural-language prompt. A small missed-token count may not justify engineering work if the total request cost, latency profile, and output quality are already acceptable.

Why a diagnostic hit can still contain uncached input tokens

A diagnostic hit is not a full-cache guarantee because the comparison checks for a detected miss against the selected recent response; it does not assert that every token in the current request had an existing cache entry. New user input, newly appended conversation turns, timestamps, dynamic retrieval excerpts, tool results, and other suffix material can remain uncached while the stable prefix is reused. In that case, the request can be healthy: the reusable policy, tool, schema, and instruction prefix may be cached, while the fresh task-specific portion is billed as ordinary input.

Another source of confusion is that comparison_response_id selects a diagnostic baseline; OpenAI states that it does not load the prior content and does not restrict which matching cache may be used. A request can compare cleanly against one baseline while actual caching is satisfied by another eligible cache entry, or it can fail to compare because the diagnostic record is unavailable while ordinary prompt caching still behaves normally. The decision rule is simple: use the diagnostic result to identify likely drift, then use usage.input_tokens_details.cached_tokens and invoice-grade usage records to measure realized reuse.

Build cost baselines that survive prompt, model, and routing changes

A cost baseline should include at least three measured cases: a cold or first-run request for the prompt family, a same-shape repeat request expected to reuse the stable prefix, and a deliberately changed request that represents a legitimate production variation. Record input tokens, cached input tokens, output tokens, model, service tier, prompt cache key if used, tools signature, text format version, reasoning effort, verbosity, and whether the request used previous_response_id, comparison_response_id, both, or neither. This record lets finance and platform teams distinguish a caching regression from an intentional quality or routing change.

Recommended calculation: compute the uncached baseline as if all input tokens were charged at the applicable standard input rate for the selected model, then compute the observed request cost using the actual split between cached and uncached input tokens plus output tokens. Do not hard-code prices in test code unless your organization has a controlled rate table update process. OpenAI’s pricing and billing terms can change, and the assigned diagnostics sources do not authorize inventing a fixed rate for this tutorial. Store the rate table version or billing export reference next to each experiment so later reviewers can reproduce the comparison.

For each prompt family, calculate a reuse ratio as cached_tokens / input_tokens and a missed-opportunity ratio as cache_missed_tokens / comparison_reusable_tokens when both diagnostic estimates are available and nonzero. The first ratio measures actual reuse on the completed request. The second ratio measures how much of the diagnostic opportunity appears to have been missed against the comparison. If these ratios disagree, trust billing usage for cost reporting and use diagnostics for root-cause analysis.

The selected article explains AI cost visibility, budget governance, and FinOps practices for tracking and controlling AI spending in 2026. The complete Why 82% of Companies Can’t Track Their AI Spending: Complete Guide to AI Cost Visibility, Budget Governance, and FinOps for 2026 article provides the destination-specific detail for this section’s AI API Cost Governance decision because it is the best match for governance-level API cost controls, complementing the current article’s tactical prompt-cache cost diagnostics with organizational budget oversight.

Design a sample that can detect real regressions

A single cache miss after a deploy is not enough to justify a rollback unless the request is high cost, high volume, or tied to a strict budget control. Design samples around prompt families rather than individual API calls. For each family, collect repeated measurements across the stable baseline, the proposed change, and a no-change control. Keep the comparison response recent and completed, because OpenAI states diagnostic records expire after a short period and comparison can return comparison_response_not_found when the record is no longer available or cannot be found for the organization.

A practical sample plan is to run five to ten repetitions for low-risk internal workflows, then increase the sample for high-volume production prompts or prompts with multiple routing branches. Vary only one dimension at a time: model, service tier, tool schema, output format, reasoning effort, verbosity, compaction strategy, or early input prefix. If your production traffic uses several prompt-cache keys, sample each key separately; otherwise, a healthy key can hide fragmentation in another key.

Stratify the sample by request shape. A short support-summary prompt, a long tool-using agent prompt, and a retrieval-heavy research prompt have different cache economics. Mixing them into one aggregate reuse number produces dashboards that look stable while specific expensive workflows regress. A useful rollout dashboard shows reuse ratio, average cached input tokens, average uncached input tokens, output tokens, diagnostic result distribution, and top first-classified miss reason by prompt family.

Handle unavailable and expired comparisons without corrupting the experiment

When diagnostics return comparison_response_not_found, first verify that the baseline response ID is recent, completed, and from the same organization. Do not automatically recreate the baseline and count the next request as a comparable data point; that changes the experiment. Instead, mark the original comparison as expired or unavailable, create a new baseline, and start a new measurement window with a new baseline identifier.

When diagnostics return unavailable, treat the request as usable for ordinary application behavior but not for miss-reason analysis. OpenAI states diagnostics are best effort and never block or fail the request. Keep the usage metrics from the response, because cached-token data can still be useful for cost analysis, but exclude the diagnostic classification from miss-reason percentages. Your dashboard should have a visible unavailable rate so teams do not mistake missing diagnostics for successful cache behavior.

Respect ZDR compatibility and record only what you need

OpenAI states that Prompt Cache Diagnostics is compatible with Zero Data Retention because it does not store raw prompts or model outputs for diagnostics; records contain organization-scoped configuration metadata, token estimates, and hashes, and expire after a short period. Your own observability pipeline should follow the same principle. Store IDs, hashes, versions, token counts, cache result fields, and classified reasons rather than raw customer prompts, documents, tool outputs, or generated text.

A safe diagnostic record should include response_id, comparison_response_id, prompt family, prompt version, model, service tier, prompt cache key, tool signature hash, text format version, reasoning effort, verbosity, input tokens, cached tokens, cache write tokens if present, comparison reusable tokens if present, cache missed tokens if present, diagnostic result, miss reason, request timestamp, deployment version, and a trace ID. If you need to debug content drift, store a salted hash of the stable prefix and separately store human-approved prompt templates in your normal secure configuration repository.

{
  "prompt_family": "policy_summarizer",
  "prompt_version": "2026-09-09.3",
  "response_id": "stored_response_id",
  "comparison_response_id": "stored_recent_completed_baseline",
  "model": "configured_supported_model",
  "service_tier": "configured_service_tier",
  "prompt_cache_key_hash": "hash_only",
  "tool_signature_hash": "hash_only",
  "text_format_version": "contract_version",
  "usage": {
    "input_tokens": 0,
    "cached_tokens": 0,
    "output_tokens": 0
  },
  "diagnostics": {
    "result": "cache_hit_or_cache_miss_or_unavailable",
    "miss_reason": "first_classified_reason_if_present",
    "comparison_reusable_tokens": 0,
    "cache_missed_tokens": 0,
    "cache_write_tokens": 0
  },
  "deployment": {
    "app_version": "release_identifier",
    "trace_id": "internal_trace_identifier"
  }
}

Define stop conditions before the rollout begins

Stop conditions prevent a cache experiment from becoming an uncontrolled production incident. Pause rollout when the same prompt family shows repeated model_changed, tools_changed, text_format_changed, or input_changed reasons after a deploy that was not supposed to touch those dimensions. Pause when actual cached_tokens drops sharply while input tokens remain stable, because that pattern points to lost reuse rather than smaller prompts. Pause when unavailable or comparison_response_not_found dominates the sample, because the diagnostic dataset can no longer support a confident root-cause decision.

Do not stop solely because reuse falls after intentional context compaction. OpenAI’s diagnostics documentation includes context_compacted as a miss reason, and compaction can lower total cost even when cache reuse declines. The correct decision is to compare total observed cost, latency requirements, output quality, and downstream acceptance. If the compacted request uses fewer total input tokens and produces acceptable results, a lower reuse ratio can be an acceptable engineering tradeoff.

Rollout dashboards that executives and engineers can both use

A production dashboard should show cache health at three levels. The executive view tracks total input tokens, cached input tokens, uncached input tokens, output tokens, and estimated cost delta versus the uncached baseline. The platform view breaks the same numbers down by model, service tier, prompt family, deployment version, and prompt cache key. The engineering view adds diagnostic result, first classified miss reason, tool signature hash, text format version, reasoning effort, verbosity, and prefix hash.

The final operating rule is conservative: diagnostics identify the first classified reason for a detected miss, while usage metrics measure actual reuse and billing systems measure spend. Keep those layers separate, compare recent completed baselines, avoid storing raw sensitive content, and promote changes only when cache behavior, total cost, and output acceptance all move in the intended direction.

Useful Links

• OpenAI Prompt Cache Diagnostics guide
• OpenAI Prompt Caching guide
• OpenAI API changelog
• OpenAI Your Data guide