GPT-6 Prompt Cache FinOps Playbook: 1.25x Writes, 0.1x Reads, Tenant Accounting, Probe Resistance, and Cost-Anomaly Evidence

Prompt-cache FinOps is a governance system, not a hit-rate contest

Prompt caching for GPT-6 changes the unit economics of repeated input, but the operational problem is larger than making a dashboard line go up. In this playbook, prompt-cache FinOps means the combined discipline of cost attribution, tenant accounting, privacy governance, routing controls, and evidence collection for cached-prefix reuse. A high cache hit rate can reduce repeated computation, but it can also obscure which tenant generated the write, which tenant benefited from the read, whether a sensitive prefix should have been cacheable at all, and whether a cost anomaly was caused by model drift, tool-schema drift, compaction, service-tier changes, or a real change in user demand.

OpenAI’s GPT-6 prompt caching announcement says eligible cached input-token reads can receive discounts of up to 90%, and the API documentation states that for GPT-5.6 and later models, cache writes cost 1.25 times the uncached input rate while cache reads cost 0.1 times the uncached input rate. Those two numbers create a simple incentive: write stable, reusable prefixes once, then read them many times. They also create a control requirement: if the application cannot attribute writes and reads to the right customer, workspace, product feature, or internal cost center, then the discount can become a billing and governance blind spot.

The practical question is not “did the request hit cache?” The practical question is “which unchanged rendered prefix was reused, under which model and settings, for which tenant or policy domain, and was that reuse allowed by the application’s data controls?” OpenAI’s prompt-caching documentation is explicit that prompt caching reuses an unchanged rendered prefix and stores key-value tensors rather than prompt tokens. It does not turn the cache into semantic memory, validate a source document, authorize sharing between users, or neutralize prompt injection. Your FinOps model has to treat cache reuse as a compute optimization with security and accounting consequences, not as a correctness signal.

This article covers migrating production systems to GPT-6 Astra, including Responses API compatibility, reasoning behavior, caching, and access-risk checks. The How to Migrate from GPT-5.5 and GPT-5.6 Sol to GPT-6 Astra: Responses API, Reasoning, Caching, and Compatibility article is a focused companion for GPT-6 Astra Prompt Caching Guide because it is the strongest available GPT-6 Astra target with explicit caching and API migration context, matching a prompt-cache FinOps playbook better than general prompt or workflow posts.

The core economics: 1.25x writes, 0.1x reads, and why the first request is more expensive

For GPT-5.6-and-later prompt caching, OpenAI documents a minimum cacheable visible prefix of 1,024 tokens. When an eligible prefix is written into cache, the cached portion is billed at 1.25x the uncached input rate. When a later compatible request reuses that cached prefix, the reused cached input is billed at 0.1x the uncached input rate. This means the first request that creates a cache entry is not automatically cheaper than ordinary processing; the savings appear only when later reads reuse enough of the written prefix.

Event	Documented multiplier for cacheable input	FinOps meaning	Governance question
Ordinary uncached input	1.0x	Baseline cost for processing input without cache reuse	Which tenant, feature, model, and request generated the input cost?
Cache write	1.25x	A premium paid to create reusable cached state for an eligible prefix	Who should be charged for creating the reusable prefix, and was the prefix approved for caching?
Cache read	0.1x	A discounted reuse of a compatible previously written prefix	Who benefited from the reuse, and did the request remain inside the same tenant, region, and authorization boundary?

A one-write/one-read example shows the break-even intuition. If a prefix would cost 1.0 input unit to process without caching, then two uncached uses would cost 2.0 units. With caching, the first use that writes the prefix costs 1.25 units, and one later full cached read costs 0.1 units, for a total of 1.35 units. On that simplified prefix-only comparison, one write plus one full read is cheaper than two uncached computations. However, that example does not include output tokens, reasoning tokens, tool calls, retries, non-cacheable suffixes, processing-mode premiums, regional premiums, or the long-context multipliers documented on the GPT-6 model pages for very large requests.

Prefix-only break-even illustration

Assume an eligible prefix has an uncached input cost of 1.00 unit.

No caching:
  Request 1 input = 1.00
  Request 2 input = 1.00
  Total = 2.00

Prompt caching:
  Request 1 cache write = 1.25
  Request 2 cache read  = 0.10
  Total = 1.35

Simplified prefix-only savings after one full read:
  2.00 - 1.35 = 0.65 input-cost units

Operational warning:
  This does not price output tokens, reasoning tokens, tools, retries,
  non-cacheable prompt suffixes, regional processing, Fast/Batch/Flex modes,
  or long-context multipliers.

The same arithmetic becomes less favorable when a prefix is written but rarely reused, when each request has a large unique suffix, or when output and reasoning tokens dominate the task cost. OpenAI’s model references list separate prices for uncached input, cached input, cache write, and output for GPT-6 Luna, Sol, and Astra; those are token prices, not total task prices. The total task bill can also be affected by tools, retries, processing mode, regional processing, and requests above the documented long-context threshold. A FinOps report that shows only “cached tokens” without output, reasoning, tools, and retry context is incomplete evidence.

This playbook explains practical engineering and CFO-grade tactics for reducing production LLM spend without degrading user experience. The AI Cost Optimization Playbook: Cut LLM Bills 80% article is a focused companion for API Cost Controls because the marker is broad API spend governance, and this target directly addresses LLM cost optimization rather than a narrower provider-specific deployment or security-control topic.

The 30-minute eligibility window is an operating assumption, not a retention guarantee

OpenAI’s API documentation currently lists `prompt_cache_options.ttl` as `30m`, and states that this is the default for GPT-5.6-and-later caching. The important interpretation is conservative: the 30-minute value is a minimum eligibility period after the latest write or reuse, while OpenAI may retain entries longer. It should not be described as a guaranteed physical-deletion deadline, a compliance retention ceiling, or a proof that sensitive content disappears from every underlying system after exactly 30 minutes.

For application owners, this distinction changes the approval model. If content is too sensitive to be cached under the organization’s policy, the right mitigation is classification and exclusion from cacheable prefixes, not relying on a short eligibility window. If a prefix contains tenant-specific policy text, proprietary documents, privileged legal material, regulated records, or confidential customer data, the application should apply explicit governance rules before allowing that prefix to become part of a reusable cache path. Human approval is required before changing tenant boundaries, retention assumptions, regional processing, sensitive-data classification, billing attribution, or incident-response policy.

Teams should also avoid interpreting a session as a cache contract. OpenAI states that a session does not guarantee a cache hit, and cache location is machine- and region-dependent. Caches are not shared across organizations or regional processing boundaries. The prompt-cache key can help with separate accounting or isolation, but OpenAI’s documentation says it does not guarantee a hit or pin a request to a particular machine. In other words, use keys and request metadata to improve attribution and reduce cross-user inference risk, not to make hard placement promises the platform documentation does not make.

Prefix reuse rewards stable architecture, but stability must not override correctness

Prompt caching reuses an unchanged rendered prefix. In practice, that means stable developer instructions, stable tool definitions, stable schemas, stable ordering, stable earlier conversation turns, stable documents, and stable media assets are more likely to benefit than prompts that rewrite the beginning of the conversation on every request. OpenAI’s GPT-6 caching announcement recommends append-only instruction and tool patterns, and the API documentation notes that changing model, tools, tool ordering, structured output schema, reasoning effort, verbosity, context management, or earlier input can cause misses.

The architectural temptation is to freeze as much as possible at the front of the prompt to maximize cache reuse. That can be a mistake. A prefix should remain stable only when it is operationally and semantically correct to keep it stable. If the policy changes, the system instruction should change. If a tool is no longer safe for a tenant, the application must not retain dangerous callability just to protect a cache line. OpenAI’s documentation suggests retaining stable tool definitions and changing callability with `allowed_tools` or `tool_choice: none` when possible, but the decision still has to respect least privilege and current authorization.

On GPT-6, OpenAI says reasoning effort can be changed through an appended `configuration_update` without rewriting the earlier cached prefix, while top-level reasoning changes can affect cache reuse. That is useful for workflows where a long stable context is followed by a higher-effort or lower-effort step. It is not a license to assume every reasoning setting is valid across every model. GPT-6 Sol and Luna document `none`, `low`, `medium`, `high`, `xhigh`, and `max`; GPT-6 Astra does not list `none`. Routing code must validate model-specific settings instead of copying one configuration across the GPT-6 family.

Prefix component	Cache-friendly pattern	Unsafe shortcut to avoid	Decision rule
Developer instructions	Keep durable policy instructions stable and append later task-specific updates	Keeping obsolete policy text because it improves hit rate	Correctness and current policy override cache reuse
Tool definitions	Preserve stable definitions and alter callability when documented and safe	Removing or reordering tools casually, then treating misses as traffic growth	Track tool schema version, ordering, and allowed-tool state in cost evidence
Tenant context	Separate tenants with accounting and isolation metadata	Sharing a prefix across tenants to increase the hit rate	Never optimize cost by weakening tenant or regional boundaries
Conversation history	Append new turns where feasible and avoid unnecessary compaction	Compacting or rewriting earlier turns without measuring cache impact	Record compaction policy changes as possible cost-anomaly causes
Reasoning configuration	Use documented GPT-6 configuration update patterns where supported	Assuming one effort value or mode works on every model	Validate against the model reference and log effort, mode, and response status

A cache hit is not correctness, authorization, or full-request reuse

A prompt-cache hit means a compatible prefix was reused. It does not mean the model’s answer is correct, the latest user request was safe, the source documents were true, the tenant was authorized to see all content, the full request was cached, or the downstream action should proceed. OpenAI’s documentation states that prompt caching reduces repeated computation; it does not validate data quality, authorize data sharing, prevent prompt injection, or replace application controls. That warning belongs in engineering design reviews, billing dashboards, and incident runbooks.

This distinction matters most in enterprise and multi-tenant systems. Suppose a legal-technology product has a stable prefix containing jurisdictional instructions, a tool schema, and a matter-specific document index. The cache may accelerate repeated analysis within the same matter, but the application still has to check user membership, matter access, jurisdictional scope, document permissions, privilege markings, and whether the requested output requires attorney review. A cached prefix can make the request cheaper; it cannot convert an unauthorized user into an authorized one or transform draft analysis into legal advice.

The same is true for educators, parents, and knowledge workers using AI systems for sensitive workflows. A cached tutoring rubric does not prove that a generated answer is age-appropriate. A cached HR policy prefix does not prove that employment guidance is compliant in a specific jurisdiction. A cached research context does not prove that cited facts are current. Current facts still require retrieval, dated sources, and verification, and consequential external messages, submissions, publications, account changes, payments, destructive actions, or legal commitments require authorized human approval.

Operational rule: Treat cache reuse as evidence about computation, not evidence about truth, permission, safety, or approval. Every high-impact workflow still needs access control, policy enforcement, audit logging, human review, rollback planning, and incident response.

Tenant accounting starts before the first cache write

Tenant accounting is not something to add after the first surprise invoice. It starts when the application decides which parts of the rendered prompt are eligible for caching and how each write and read will be attributed. At minimum, a production FinOps record should distinguish the organization, workspace, tenant, project, user or service principal category, feature, model, region or processing boundary where applicable, prompt version, tool-schema version, cache mode, cache key strategy, token categories, and response status. Without that evidence, a cost spike can look like user growth even when it was caused by a missed prefix after a schema reorder.

A conservative accounting policy separates the actor who created a cache write from the actor who benefited from later reads. In a single-tenant product, those may map to the same customer. In a multi-team enterprise workspace, the write may be triggered by one department and the read by another department using the same approved application context. In a multi-tenant SaaS product, cross-tenant reuse should not be used as a cost optimization strategy. OpenAI states caches are not shared across organizations or regional processing boundaries, and the application should preserve its own tenant and data-region boundaries regardless of any technical opportunity to increase reuse.

The cleanest FinOps ledger treats each request as a bundle of billable and governable components: uncached input tokens, cache-write tokens, cached-read tokens, output tokens, reasoning tokens where applicable, tool calls, retries, processing-mode adjustments, regional premiums, and long-context multipliers. The model pages for GPT-6 Luna, Sol, and Astra publish different token prices and document that requests above 272,000 input tokens have multipliers for the full request. A “cost per request” average that hides these components will not explain anomalies when a workload crosses the long-context threshold or changes from Luna to Sol or Astra.

Recommended cache FinOps event fields

request_id
timestamp
organization_id_or_internal_org_reference
workspace_or_account_reference
tenant_reference
cost_center
application_feature
environment
model_id
processing_mode_if_recorded_by_application
region_or_data_residency_policy_if_applicable
prompt_version
rendered_prefix_version
tool_schema_version
tool_order_hash
structured_output_schema_version
reasoning_effort
reasoning_mode_if_applicable
verbosity_or_text_format_setting_if_applicable
context_management_or_compaction_policy
prompt_cache_mode
prompt_cache_key_or_key_family
cache_write_tokens
cached_read_tokens
uncached_input_tokens
output_tokens
reasoning_tokens_if_reported_in_usage
tool_call_count
retry_count
response_status
incomplete_reason_if_applicable
human_approval_required
human_approval_status
billing_attribution_policy_version
sensitive_content_classification
incident_or_anomaly_reference

Probe resistance: prevent cache behavior from becoming a side channel

Prompt-cache behavior can leak operational hints if teams expose too much detail to untrusted users. A malicious or curious user should not be able to infer that another tenant recently used a particular document, tool schema, legal matter, customer name, or investigation package by measuring latency, token accounting, or cache diagnostics. OpenAI’s documentation notes that separate cache accounting can reduce cross-user cache-hit probing risks. In application terms, probe resistance means minimizing externally visible cache signals and ensuring cache keys, logs, dashboards, and billing views do not reveal another party’s activity.

The first control is tenant separation. Do not design cross-tenant prefix sharing to raise hit rate. Do not prewarm shared caches with customer-specific documents. Do not show one tenant cache-hit evidence that was generated by another tenant’s write. If a product offers customer billing reports, report the customer’s own usage and discounts according to a transparent policy, not raw cache events that reveal timing or behavior from other tenants. Internal engineers may need detailed evidence for debugging, but access to that evidence should be role-limited, logged, and reviewed like other sensitive operational telemetry.

The second control is diagnostic containment. OpenAI’s prompt-cache diagnostics can compare a request against a recent completed response by setting `prompt_cache_options.comparison_response_id`. The documentation states that this asks for diagnostic metadata, does not load the earlier conversation, and does not change caching behavior. Diagnostic results are best effort, may return `cache_hit`, `cache_miss`, `comparison_response_not_found`, or `unavailable`, and actual reuse should be measured through `usage.input_tokens_details.cached_tokens`. User-facing applications should not expose raw diagnostic reasons to arbitrary end users because reasons such as tools, schema, model, service tier, or input changes can reveal system design details.

The third control is logging discipline. OpenAI states that ZDR-compatible diagnostics do not store raw prompts or outputs for the diagnostic feature and instead use configuration metadata, token-count estimates, and hashes. That statement does not automatically cover your own application logs, analytics pipelines, observability vendors, support exports, or billing warehouse. If your application logs raw prompts, documents, tool arguments, or outputs, those logs can create privacy and retention risks independent of OpenAI’s diagnostic design. Review logging, redaction, retention, and access controls separately.

Opening operating model: classify, attribute, measure, and approve

The safest starting point is a four-part operating model: classify cacheable content, attribute writes and reads, measure reuse and anomalies, and require approval for boundary changes. Classification decides which prompt components may be cached, which must be excluded or minimized, and which require special tenant, region, or role controls. Attribution decides who pays for writes, who receives read discounts, and how shared internal prefixes are charged. Measurement distinguishes true demand growth from cache misses caused by drift. Approval prevents cost optimization from silently changing privacy, billing, retention, regional, or authorization policy.

1. Classify prefixes before rollout. Separate global product instructions, tenant-neutral tool schemas, tenant-specific configuration, customer documents, personal data, regulated records, secrets, privileged material, and externally sourced content. Do not put secrets, tokens, private keys, or unnecessary confidential material into prompts or prewarming jobs.
2. Define attribution rules. Decide whether cache writes are charged to the first requester, amortized across a tenant, absorbed by the platform, or allocated to a cost center. Decide how read discounts are represented to customers and internal teams without hiding material cache costs.
3. Record settings that affect reuse. Log model, tools, tool ordering, schema, service tier where applicable, reasoning effort, verbosity, compaction, prompt version, cache mode, and cache key family. These are known causes of cache misses in OpenAI’s documentation and diagnostics.
4. Use diagnostics as evidence, not truth. Treat diagnostic reasons as best-effort clues. Confirm actual reuse with cached-token usage metrics, request diffs, deployment records, and billing data.
5. Require human approval for consequential changes. Boundary changes involving tenants, retention, regional processing, sensitive-data classification, billing attribution, incident response, publication, external messaging, legal commitments, payments, purchases, or destructive actions should not be automated solely because cache metrics look favorable.

A mature cache FinOps program should be able to answer three questions during an incident or billing review. First, what changed: model, prompt, schema, tool order, service tier, reasoning effort, verbosity, compaction, traffic mix, long-context share, or retry rate? Second, who was affected: which tenants, workspaces, products, regions, and cost centers saw write spikes, read drops, or output-token growth? Third, what evidence supports the conclusion: usage token details, diagnostic metadata, deployment logs, prompt-version diffs, approval records, and rollback timestamps? If the team cannot answer those questions, it does not yet have prompt-cache FinOps; it has a cache feature with incomplete governance.

What this playbook will build

The rest of this playbook turns those principles into an implementable operating system for GPT-6 prompt caching. It will cover explicit versus implicit caching modes, cacheable-prefix inventory, break-even math by model family member, tenant and cost-center ledgers, cache-key strategy, long-context multipliers, drift detection, diagnostic workflows, dashboard design, anomaly evidence, prewarming approvals, Zero Data Retention considerations, regional boundaries, incident response, and rollback. The goal is not to maximize cache hits at any cost; the goal is to reduce repeated computation while preserving authorization, privacy, billing accuracy, auditability, and human accountability.

The key design posture is defensive. Prompt caching can materially reduce repeated input cost when stable prefixes are reused, and OpenAI’s GPT-6 changes give developers more control through higher default hit rates, diagnostics, explicit breakpoints, prewarming, and append-only patterns. But every optimization must remain subordinate to application-specific evaluation, least privilege, tenant isolation, data-region rules, and appropriate human review. A lower-cost cached route never authorizes broader tool access, weaker logging, cross-tenant data reuse, or unattended high-impact action.

Build the cacheable-prefix inventory before choosing a caching mode

A GPT-6 prompt-cache program should start with an inventory of prefixes that are both technically reusable and governance-approved for reuse within the documented prompt-caching model. OpenAI’s prompt-caching documentation says supported models can reuse an unchanged rendered prefix and that cached state can cover developer messages, tool definitions, conversation history, text, images, documents, and supported audio. That breadth is useful, but it also means a “prefix” is not merely the first visible paragraph in a prompt; it is the rendered request material that reaches the model before the variable tail.

Recommendation: maintain a cacheable-prefix register as a controlled engineering artifact, not a spreadsheet discovered after billing anomalies occur. Each candidate prefix should have an owner, a prompt version, a model family, a data classification, an approved tenant scope, a regional-processing expectation, a mode decision, and a prewarming decision. If the prefix cannot be described in those terms, it is not mature enough to treat as a FinOps asset.

Inventory field	Why it matters	Decision rule
Prefix ID and version	Connects spend, hit-rate evidence, diagnostics, and rollback to a known prompt artifact.	Use immutable versions for deployed prefixes; create a new version when earlier content, tool definitions, schemas, or ordering changes.
Model ID	Cache compatibility can depend on the model. GPT-6 API identifiers include model-specific references such as gpt-6-luna, gpt-6-sol, and gpt-6-astra.	Do not assume that a prefix validated for one GPT-6 model has equivalent economics or behavior on another.
Endpoint and feature surface	The cited model pages list supported endpoints and tools, while some older endpoints are not supported for these models.	Record whether the workload uses Responses, Chat Completions, Batch, streaming, structured outputs, function calling, file search, image input, web search, or another supported capability.
Stable developer instructions	Stable earlier instructions are strong cache candidates, but stale or overbroad instructions can create correctness and authorization problems.	Cache only instructions that remain semantically correct for the tenant, workflow, model, and tool policy.
Tool definitions and schema order	OpenAI states that tools, tool ordering, and structured output schema changes can affect cache reuse.	Keep tool definitions and schemas stable where correct; use availability controls such as allowed_tools or tool_choice: none rather than removing definitions solely to chase a hit rate.
Variable tail boundary	The boundary separates reusable prompt material from user-specific, task-specific, or sensitive content that should not be generalized.	Place tenant-specific data, user requests, retrieved records, secrets, and case facts after the approved stable prefix unless a security review explicitly approves their placement.
Data classification	Prompt caching does not authorize data sharing or validate whether content is appropriate to cache.	Classify before deployment as public, internal, confidential, regulated, privileged, youth-related, security-sensitive, or prohibited for caching/prewarming.
Tenant and region scope	OpenAI documents that caches are not shared across organizations or regional processing boundaries, and cache location is machine- and region-dependent.	Do not design cross-tenant prefix sharing. Do not rely on cache behavior to bridge regions or organizations.
Accounting key strategy	prompt_cache_key is not required for GPT-5.6 and later routing but may be useful for accounting or isolation.	Use keys to support tenant-level chargeback and probe resistance where helpful; never treat a key as a machine pinning or hit guarantee.
Prewarming status	Prewarming can create cache writes before live demand, increasing spend and governance exposure.	Require approval for every prewarm plan, and prohibit secret-bearing or cross-tenant prewarming.

The inventory should include the approximate token length of each stable prefix and the variable tail that normally follows it. OpenAI documents a minimum cacheable visible prefix of 1,024 tokens for GPT-5.6 and later. That minimum should not encourage teams to pad prompts; it should instead help FinOps teams reject tiny prefixes that cannot become eligible and prioritize larger, stable, frequently reused prefixes where the 1.25x write and 0.1x read economics can matter.

Operational warning: a long prefix can be expensive even when cached. The GPT-6 model references state that requests above 272,000 input tokens have documented multipliers for the full request. If a team celebrates cached-token discounts without accounting for long-context multipliers, output tokens, tool charges, processing mode, regional premium, and retries, the resulting “savings” analysis will be incomplete.

Separate reusable instructions from reusable data

The safest early wins usually come from stable developer instructions, tool declarations, schema definitions, rubric text, and workflow policy that are identical across many requests within a tenant-controlled boundary. These prefixes tend to be easier to approve because they do not contain customer records, employee records, privileged facts, secrets, health information, legal matter details, or live account data.

Reusable data deserves a higher bar. A knowledge excerpt, document bundle, conversation history, image, or file-derived context may be technically cacheable under OpenAI’s documentation, but technical eligibility is not the same as authorization. Before allowing such content into an approved prefix, require a written classification decision that explains who owns the content, which tenant may reuse it, which users may cause it to be sent, whether regional processing constraints apply, and whether any retention, contractual, privilege, or Zero Data Retention commitments are implicated.

Example inventory row: a legal-technology vendor may approve a “motion-drafting style guide and output schema” prefix for one enterprise tenant’s litigation workspace, while rejecting a proposed prefix that includes prior case facts or attorney-client communications. The former can be stable and tenant-scoped; the latter may be matter-specific, privileged, and unsuitable for prewarming or broad reuse even if it would technically improve hit rates.

This playbook treats artifacts, files, logs, summaries, and handoff notes as controlled objects, with approved transfers, repository boundaries, egress gates, and incident evidence for Codex-style workflows. The Codex Artifact and Multi-Agent Isolation Playbook: Approved Transfers, Repository Boundaries, Egress Gates, and Incident Evidence article is a focused companion for Multi Tenant Data Isolation because tenant accounting and cache isolation depend on preventing unauthorized cross-boundary movement of data or artifacts, making this the most relevant isolation-focused candidate.

Choose explicit or implicit mode with a documented failure policy

OpenAI describes prompt caching as enabled by default for supported models and documents both explicit and implicit modes. In implicit operation, compatible unchanged prefixes can be reused without the application marking a breakpoint. In explicit-only mode, no breakpoint means no prompt caching. That distinction is a governance choice, not merely a performance tuning preference.

Mode decision	Best fit	Primary risk	Required control
Implicit caching	Low-risk stable prefixes where the application already has strong tenant routing, logging, and prompt versioning.	Developers may not notice that a prefix changed enough to reduce reuse or that eligible material includes content that should have been isolated.	Automated accounting by model, tenant, prefix version, cache key, and cached-token count.
Explicit breakpoints	Workflows with deliberate stable-prefix boundaries, tool schemas, and high-volume repeated prefixes.	Incorrect breakpoint placement may include sensitive variable material or exclude the most reusable stable material.	Code review of the rendered prompt boundary and security review for sensitive data placement.
Explicit-only caching	Regulated, multi-tenant, legal, health, education, financial, or enterprise-administration workflows that require positive approval before any cache write.	Forgetting a breakpoint disables caching and can raise cost, but that failure is often safer than accidental caching.	Deployment tests that assert approved breakpoints exist for approved prefixes and do not exist for prohibited prefixes.
Caching disabled or not used for a workflow	Highly sensitive, one-off, secret-bearing, incident-response, privileged, or regionally constrained workflows where reuse has little benefit or unresolved governance risk.	Higher input processing cost for repeated material.	Documented exception with cost owner approval, not silent bypass by individual developers.

Decision rule: use implicit caching only when accidental eligibility would not violate tenant boundaries, sensitivity rules, or regional expectations. Use explicit breakpoints or explicit-only mode when the organization needs an auditable statement that “only this approved prefix is eligible.” Use no caching or no prewarming for secret-bearing, privileged, or unusually sensitive workloads unless a qualified privacy, security, and legal review approves the exact design.

In explicit mode, remember OpenAI’s four-write limit: each request can create up to four cache writes. That limit should force discipline. Treat each write as an intentional boundary: a base instruction block, a stable tool/schema block, a tenant-approved policy block, and perhaps a workflow-specific rubric. If a design needs many more writes, the prompt architecture may be trying to cache unstable or over-fragmented material.

Operational warning: the four-write limit is not a reason to combine unrelated tenants, workspaces, matters, classes, patients, accounts, or projects into one massive prefix. Combining unrelated material may improve apparent reuse but can break isolation, confuse chargeback, increase privacy risk, and create harder incident investigations. A cache optimization that weakens authorization is not a valid FinOps control.

Breakpoint placement procedure

1. Render the final request exactly as sent. Review the actual developer messages, tool definitions, schemas, conversation material, files, and any generated system or application scaffolding, not only the template source.
2. Mark the earliest stable boundary. Identify content that is unchanged across repeated requests and semantically safe to reuse for the approved tenant scope.
3. Stop before tenant-variable facts. User requests, retrieved documents, tickets, health facts, student records, legal matter details, credentials, account identifiers, and confidential notes should not be pulled into a reusable boundary by default.
4. Check tool/schema stability. OpenAI notes that tools, tool ordering, and structured output schemas can affect reuse; changes here should produce a new prefix version.
5. Validate compatible settings. Cache misses can result from model, service tier, reasoning effort, verbosity, context management, compaction, and earlier input changes. Record those settings alongside the prefix version.
6. Estimate economics before deployment. A write costs 1.25x the uncached input rate and a read costs 0.1x for GPT-5.6 and later, but output tokens, reasoning tokens, tool calls, long-context multipliers, regional processing, and retries remain separate.
7. Obtain approval for sensitive classes. Human approval is required before changing tenant boundaries, sensitive-data classification, billing attribution, regional processing, retention interpretation, or incident-response handling.

Design stable prefixes that tolerate change without rewriting the past

OpenAI’s GPT-6 caching announcement emphasizes append-only instruction and tool patterns. The practical implication is simple: design the prefix so stable material remains stable, and put changing decisions later in the context. Rewriting an early policy paragraph, moving tool definitions, changing schema order, or compacting earlier conversation can reduce cache reuse even if the visible task seems the same to the product team.

Recommended structure: place stable developer policy first, then stable tool definitions and structured output schemas, then tenant-approved static context, then appended configuration or task-specific instructions, then user-specific or retrieved material. On GPT-6, OpenAI states that reasoning effort can be changed through an appended configuration_update without rewriting the earlier cached prefix, while top-level reasoning changes can affect cache reuse. Applications should still verify support against the relevant model documentation and observe actual usage metrics.

{
  "prefix_design_policy": {
    "stable_segment_1": "Developer instructions approved for this workflow and tenant scope",
    "stable_segment_2": "Tool definitions, schema definitions, and ordering that remain unchanged",
    "stable_segment_3": "Tenant-approved static policy or rubric, if allowed by classification",
    "later_segment_1": "Configuration update or task-specific instruction appended without rewriting prior prefix",
    "later_segment_2": "User request, retrieved records, and other variable content"
  },
  "governance_rules": [
    "Do not insert secrets into stable segments",
    "Do not combine tenants to increase reuse",
    "Do not move sensitive variable data ahead of the breakpoint",
    "Create a new prefix version when stable content changes"
  ]
}

The design should also avoid “prompt compaction” that rewrites earlier turns merely to reduce context size. OpenAI lists compaction as a possible diagnostic miss reason. If a product uses conversation summarization, memory pruning, transcript rewriting, or dynamic policy injection, those transformations should be accounted for as cache-affecting behavior and tested against representative traffic.

Example: a coding assistant that repeatedly sends the same repository-level rules, tool definitions, and output schema can keep those early sections stable while appending the specific issue, diff, and review request later. If the application removes a tool definition whenever a user lacks permission, the rendered tool list changes and may reduce reuse; a safer pattern, when supported and semantically correct, is to keep definitions stable and restrict callability with an allowed-tools mechanism or by selecting no tool use for that request.

Stability should never override permissions. If a user is not allowed to use a tool, access a file, view a document, or trigger an external operation, the application must enforce that restriction even if enforcement reduces cache reuse. Cost is not an authorization layer, and a lower cached-input rate does not justify broader tool scope.

Build tenant accounting around writes, reads, misses, and shared infrastructure costs

A usable chargeback model separates cache writes from cached reads, uncached input, output, tool costs, retries, processing modes, and regional premiums. OpenAI’s GPT-6 model references publish separate prices for uncached input, cached input, cache writes, and output for Luna, Sol, and Astra. Those prices are token prices, not total task prices. A chargeback system that only records total tokens will not explain whether cost increased because of more writes, fewer reads, longer outputs, higher reasoning effort, tool calls, a processing-mode change, or long-context multipliers.

Chargeback field	Purpose	Example value type
Organization ID or billing account	Confirms the commercial boundary. OpenAI states caches are not shared across organizations.	Internal billing identifier, not a secret or API key.
Tenant ID	Attributes cost to the customer, workspace, school, department, or legal matter owner.	Pseudonymous tenant key controlled by the application.
User or service principal class	Distinguishes human user traffic, background jobs, prewarming jobs, and administrative tests.	Role category rather than unnecessary personal data.
Model ID	Separates Luna, Sol, Astra, and any future model-specific pricing or behavior.	gpt-6-luna, gpt-6-sol, or gpt-6-astra.
Endpoint and processing mode	Batch, Flex, Fast, Standard, regional processing, and endpoint differences can change effective cost and behavior.	Responses Standard, Batch, Fast, or an internal enum aligned with official availability.
Region or data residency class	Supports regional processing boundaries and premium analysis where available.	Approved region class; avoid storing unnecessary location detail.
Prompt prefix ID and version	Links hits, misses, writes, and anomalies to a specific approved prompt artifact.	contract_review_v17 or an internal immutable version identifier.
prompt_cache_key	Supports separate accounting or isolation where used.	Hashed or pseudonymous key; not a tenant secret.
Uncached input tokens	Measures fresh input processing.	Numeric token count from response usage records.
Cache-write tokens	Measures input tokens charged at the documented write rate.	Numeric token count or derived field based on usage details and billing export.
Cached-read tokens	Measures reused prefix input charged at the cached-input rate.	usage.input_tokens_details.cached_tokens where available.
Output and reasoning tokens	Reasoning tokens are billable output tokens even when not visible, according to OpenAI’s reasoning documentation.	Numeric output-token fields from usage records.
Tool-call charges and retries	Separates model cost from external tools, hosted tools, failed attempts, and application retry storms.	Tool name class, count, and billable units where available.
Diagnostic comparison ID used	Connects miss investigations to best-effort diagnostic metadata.	Recent completed response ID reference stored under access control.
Approval reference	Shows that sensitive prefix, prewarming, regional, or billing decisions were authorized.	Ticket, change request, or governance record number.

Accounting rule: charge the write to the tenant or internal platform owner that caused the write, and charge reads to the tenant or job that benefits from reuse. When a platform team deliberately prewarms a prefix for a tenant, the prewarm write should be visible as a platform-initiated cost, not hidden inside an end user’s task. When a prefix is used by multiple departments inside the same tenant boundary, allocate reads according to actual usage rather than flattening the cost into a vague “AI overhead” pool.

Do not cross-charge one tenant for another tenant’s prefix economics. Even if two tenants use identical public instructions, the application should not depend on cross-tenant cache sharing, should not present one tenant’s prewarm as available to another, and should not make cost allocation contingent on reuse that crosses tenant boundaries. OpenAI’s documentation already states caches are not shared across organizations, and a well-designed multi-tenant system should maintain its own isolation and audit boundaries inside an organization as well.

Optional cache keys: accounting aid, not a routing guarantee

OpenAI states that prompt_cache_key is not required for GPT-5.6 and later routing, but may be used for separate accounting or isolation. It also states that keys do not guarantee a hit or pin a request to a machine. That means a cache key is best understood as a governance and observability signal, not as a mechanical promise.

Recommended key design: derive keys from non-secret, pseudonymous governance dimensions such as tenant, workspace, prefix version, region class, and workload class. Avoid raw customer names, email addresses, matter names, account numbers, access tokens, private repository names, or document titles. A key should be safe enough to appear in restricted billing logs and diagnostic workflows without exposing confidential content.

{
  "prompt_cache_key_strategy": {
    "include": [
      "tenant_pseudonym",
      "workspace_pseudonym",
      "prefix_version",
      "approved_region_class",
      "workload_class"
    ],
    "exclude": [
      "passwords",
      "API keys",
      "private keys",
      "personal identifiers",
      "case names",
      "patient or student identifiers",
      "repository secrets",
      "raw document titles"
    ],
    "interpretation": "Accounting and isolation signal only; not a hit guarantee or machine pin."
  }
}

Probe-resistance benefit: separate accounting keys can reduce cross-user cache-hit probing risk because users or jobs are less likely to influence or observe each other’s cache state through timing, token accounting, or differential cost patterns. This benefit is partial. It does not replace access control, rate limiting, tenant isolation, redaction, logging discipline, or human review for sensitive boundary changes.

Classify sensitive content before it can enter a cacheable prefix

Prompt caching reduces repeated computation. It does not decide whether content is safe to send, safe to cache, safe to prewarm, safe to log, or safe to reuse. OpenAI’s documentation notes that cached states may be stored as encrypted application state in GPU-local storage and that availability and retention settings vary by model and organization policy. Because those details are not a universal privacy guarantee, enterprises should classify content before it is placed into any stable prefix or prewarm job.

Classification	Caching posture	Prewarming posture	Required review
Public product instructions or public documentation excerpts	Usually suitable if accurate, stable, and licensed or authorized for the use case.	May be suitable when economics justify it.	Product owner and FinOps review.
Internal workflow policy	Suitable only within the approved organization, tenant, workspace, and role boundary.	Possible, but only for approved tenant/workspace scopes.	Security and data owner review.
Confidential customer data	Default to not placing in stable prefixes unless explicitly approved for that tenant and workflow.	Do not prewarm by default.	Privacy, security, contract, and data owner review.
Regulated records such as health, financial, student, employment, or youth-related records	Highly restricted; cache only if the full compliance design permits it.	Do not prewarm unless an authorized compliance decision specifically approves the exact content and purpose.	Qualified compliance, legal, privacy, and security review.
Privileged legal material or confidential legal matter facts	Default to no reusable prefix unless matter-specific controls and privilege analysis approve it.	Do not prewarm secret-bearing or privileged facts.	Authorized legal and security review.
Secrets, credentials, private keys, session tokens, exploit details, or sensitive security telemetry	Do not place in cacheable stable prefixes.	Prohibited.	Security incident process if exposed.

Policy proposal: require a “cache classification” field in every prompt-template pull request. Acceptable values should be narrow: cache-approved-public, cache-approved-internal, cache-approved-tenant-confidential, cache-restricted, and cache-prohibited. The pull request should fail if the field is missing or if a restricted/prohibited prefix has an explicit breakpoint or prewarming job attached without an approval reference.

Legal-technology and education workflows need additional caution because the same stable wording can carry different confidentiality implications across matters or students. A generic citation-formatting rubric may be cacheable. A set of facts from a juvenile disciplinary record, a settlement memo, a student accommodation file, or a privileged litigation timeline should not become a broad prefix merely because it appears in multiple requests for the same user.

Reduce cross-user probe risk with isolation, aggregation, and evidence discipline

Cache-hit probing occurs when a user or workload can infer that someone else recently sent similar content by observing latency, usage details, cached-token counts, diagnostics, or cost differences. OpenAI’s documentation warns that separate cache accounting can reduce cross-user cache-hit probing risks. The application still has to design the user-facing surface so cache behavior does not become a side channel.

Recommended controls: keep detailed cache metrics in restricted operational logs, show user-facing cost only at safe aggregation levels, avoid exposing raw cached-token counts to untrusted users, and do not let one user supply arbitrary prefixes for the purpose of testing whether another user’s content is cached. If a product has per-request cost transparency, use rounding, delay, aggregation, or tenant-scoped reporting where appropriate so the display supports budgeting without revealing another user’s activity.

1. Partition by tenant and approved workspace. Do not allow users from separate tenants, schools, customers, departments, or legal matters to intentionally share cacheable prefixes.
2. Use optional cache keys where helpful. Apply pseudonymous keys for tenant/workspace/prefix isolation and accounting, while remembering that keys do not guarantee hits.
3. Restrict diagnostics access. Diagnostics can reveal miss reasons such as model, cache key, tools, reasoning effort, verbosity, compaction, and input changes. Treat that metadata as operational evidence, not as end-user content.
4. Minimize timing signals. Avoid product experiences where an untrusted user can repeatedly submit probes and see high-resolution timing differences linked to cache state.
5. Rate-limit suspicious comparison attempts. A workload that repeatedly varies early input to test cache behavior should be investigated like other enumeration behavior.
6. Separate billing visibility from raw cache internals. Customers can receive fair chargeback without receiving fields that would expose another user’s recent activity.

Operational warning: diagnostics are not a safe oracle for proving that a user’s guessed text matched another user’s request. OpenAI’s diagnostics are best effort, return the first classified reason, and a diagnostic cache_hit means no comparison miss was detected; it does not mean the whole current prompt came from cache. Treat diagnostics as engineering evidence for your own recent requests, not as a feature for user-visible comparison or content discovery.

Respect regional processing boundaries and review Zero Data Retention assumptions

OpenAI’s prompt-caching documentation states that cache location is machine- and region-dependent and that caches are not shared across organizations or regional processing boundaries. The GPT-6 model references also note regional processing premiums where available for Sol and Luna and identify EU data residency availability constraints for those models. A FinOps plan must therefore track region and processing mode as first-class cost and compliance dimensions rather than assuming that a cache created in one processing context will improve economics in another.

Regional boundary rule: if a tenant, contract, workspace, or dataset requires a particular processing region or residency posture, the application must route requests according to that rule even when another route would be cheaper or more cache-efficient. Cost-only routing must not override data-region commitments. Human approval is required for changes to regional processing, residency interpretation, tenant boundary, or sensitive-data classification.

Zero Data Retention also requires careful interpretation. OpenAI’s prompt-cache diagnostics documentation states that diagnostics are compatible with Zero Data Retention and store configuration metadata, token-count estimates, and hashes rather than raw prompts or outputs for the diagnostic feature. That statement should not be expanded into a blanket claim that every application log, tool call, file store, analytics system, support workflow, or customer dashboard has the same retention behavior.

ZDR review checklist: confirm the model and organization policy, confirm whether prompt caching is available under the tenant’s policy, confirm what the application itself logs, confirm whether diagnostics are enabled and who can access diagnostic metadata, confirm whether tool calls store separate data, and confirm whether prewarming jobs create additional records in orchestration, queue, billing, or monitoring systems. If any part of the request path stores raw prompts or outputs outside the model diagnostic feature, it needs its own retention and access-control review.

Governance interpretation: the documented 30-minute ttl value is a minimum eligibility period after the latest write or reuse for GPT-5.6-and-later caching under the current setting, not a guaranteed physical-retention ceiling. Do not promise customers, students, employees, or regulated users that cached state is physically deleted at exactly 30 minutes unless an official contractual and technical commitment separately supports that statement.

Retention language should be precise in customer-facing documentation. A safe statement is that the application uses the provider’s documented prompt-caching controls and treats the 30-minute value as an eligibility window for reuse, while separate contractual terms, organization policies, logging settings, and support processes determine broader retention obligations. Do not translate “minimum eligibility window” into “maximum storage duration.”

Approve prewarming only when the content, tenant, region, and cost owner are clear

OpenAI’s GPT-6 caching announcement describes prewarming as one of the controls developers can use to improve cache behavior. Prewarming is optional. It creates cost before a live user request benefits from reuse, and it can send content through model infrastructure without an immediate user-facing task. That makes prewarming a governance workflow, not a background optimization that an engineer should silently enable.

Approval requirement: every prewarming job should identify the prefix version, model, tenant/workspace scope, region or processing mode, expected traffic pattern, cost owner, data classification, and rollback plan. Human approval is mandatory before prewarming sensitive prefixes, changing billing attribution, changing regional processing, or changing tenant isolation. Secret-bearing prewarming should be prohibited because prewarming does not validate, authorize, or secure the content.

Prewarming question	Required answer before approval
What exact prefix will be prewarmed?	An immutable prefix version with rendered-content review and classification.
Which tenant or workspace benefits?	A specific approved scope; not “all tenants” and not a cross-tenant pool.
What content class is included?	Public, internal, or explicitly approved tenant-confidential content; no credentials, secrets, or privileged facts by default.
What region and processing mode apply?	The same approved boundary that production traffic must use.
Who pays for writes that do not get reused?	A named platform, department, or tenant cost center with budget authority.
What hit-rate and cost evidence justifies the job?	Representative traffic estimates and post-launch measurement using cached-token fields, not vendor examples alone.
What stops the job?	Budget threshold, anomaly threshold, prefix version replacement, tenant offboarding, region change, or incident response.

Prewarming should have an expiration or renewal process. A prefix that was valuable during a quarterly reporting cycle, admissions period, code freeze, litigation deadline, or customer onboarding wave may become wasteful later. Because cache writes cost 1.25x the uncached input rate, prewarming without reuse can be more expensive than doing nothing.

Example approval note: “Approve prewarming for prefix support_triage_schema_v09 for Tenant A’s North America support workspace using the approved processing mode for that tenant. Content contains internal workflow instructions and a stable structured output schema only. No customer tickets, credentials, attachments, or account identifiers are included. Platform FinOps owns prewarm writes. Job stops after the campaign window or if cached-read tokens fall below the approved threshold for two consecutive measurement windows.”

That example is a governance pattern, not a universal policy. Each organization must align the approval path with its contracts, data classifications, regional commitments, security controls, and finance model. The non-negotiable principle is that prewarming must not become a way to smuggle sensitive or cross-tenant content into a reusable prefix for cost reasons.

Monitor cache economics with measures that match the billable token paths

A useful prompt-cache dashboard must separate what was processed as ordinary input, what was written to cache at the cache-write rate, what was read from cache at the cached-input rate, what was generated as output, and what was spent on tools, processing mode, regional processing, retries, or long-context multipliers. OpenAI’s prompt-caching documentation says GPT-5.6-and-later cache writes cost 1.25 times the uncached input rate and cached reads cost 0.1 times the uncached input rate. OpenAI’s GPT-6 model reference pages publish model-specific Standard API prices for Luna, Sol, and Astra, and those pages also document additional pricing effects such as long-context multipliers above 272,000 input tokens, regional processing premiums where available, Batch/Flex pricing, Fast mode pricing, and possible separate tool charges. A dashboard that shows only “total tokens” or “cache hit rate” will hide the causal evidence required for FinOps review.

Define one immutable cost ledger row per model request, not one row per user-visible task. A user-visible task may include a prewarm call, a planning call, several tool-using calls, a retry after an incomplete response, and a final summarization call. Each request can have a different model, service tier, region, prompt-cache key, explicit-breakpoint set, tool definition set, schema, reasoning effort, verbosity, and context size. Aggregating these into a single blended task cost is useful for executives, but the raw ledger must preserve request-level fields so engineering and finance can explain why a cache write, read, miss, or output burst occurred.

Dashboard measure	Operational definition	Why it matters	Review warning
Actual cached tokens	Tokens reported as reused through usage.input_tokens_details.cached_tokens for the completed request.	OpenAI’s diagnostics guide states actual reuse is measured through cached tokens, not by assuming a diagnostic hit means the full prompt was cached.	A diagnostic cache_hit means no comparison miss was detected; it is not proof that every current input token came from cache.
Uncached input tokens	Visible input tokens billed at the ordinary model input rate, after subtracting actual cached-token usage and separately accounting for write tokens when reported or estimated by your ledger design.	Uncached input is the baseline against which write and read economics are compared.	Do not hide uncached growth behind a rising hit rate; a larger prompt can have more cached tokens and still cost more.
Cache-write tokens	Tokens attributable to a cache write event under the request’s prompt-cache configuration, costed at 1.25x ordinary input for GPT-5.6-and-later caching.	Writes are intentionally more expensive than ordinary input, so uncontrolled prewarming or one-off writes can increase spend.	Each request can create up to four cache writes under the documented prompt-caching behavior; monitor write count and write placement.
Cached-read tokens	Tokens reused from an eligible unchanged prefix, costed at 0.1x ordinary input for GPT-5.6-and-later caching.	Cached reads are the main savings mechanism for repeated stable prefixes.	Reads are prefix reuse, not semantic validation, permission approval, or data-sharing authorization.
Output and reasoning tokens	All output-billed tokens, including invisible reasoning tokens where applicable, costed at the selected model’s output rate.	Prompt caching reduces repeated input computation; it does not discount generated output or reasoning work.	A cached-prefix optimization can be overwhelmed by higher reasoning effort, pro mode, verbose output, or retries.
Tool cost	Separate charges or metered usage associated with enabled tools, tool calls, and tool-side infrastructure.	OpenAI’s model pages state tool calls may add separate charges, so token-only dashboards are incomplete.	Do not route to cheaper input tokens while silently increasing tool authority, tool frequency, or external side effects.
Long-context multiplier flag	Boolean and multiplier fields for requests above 272,000 input tokens, where OpenAI documents 2x input and cache rates and 1.5x output rates for the full request.	Large contexts can invert apparent savings because the multiplier applies to the full request, not only the marginal excess.	Never compare a 300,000-token request to a 100,000-token request using list prices alone.

For GPT-6 Luna, Sol, and Astra, preserve model-specific list prices in a reference table that is versioned by date and source. OpenAI’s model reference pages list Luna Standard API prices per one million tokens as $0.10 input, $0.01 cached input, $0.125 cache write, and $0.50 output; Sol as $2 input, $0.20 cached input, $2.50 cache write, and $10 output; and Astra as $10 input, $1 cached input, $12.50 cache write, and $50 output. These values should be used as explicit rate-card inputs, not embedded as magic constants in application code, because plan, mode, region, and future source updates can change the effective bill.

Use a cost formula that keeps the billable paths visible. The formula below is a recommended FinOps model, not an OpenAI endpoint contract. It is designed to force reviewers to include cache writes, cached reads, uncached input, output, long-context effects, tools, regional processing, service tier, and retries as separate variables.

request_cost =
  long_context_input_multiplier *
    (
      uncached_input_tokens * model_input_rate
      + cache_write_tokens * model_cache_write_rate
      + cached_read_tokens * model_cached_input_rate
    )
  + long_context_output_multiplier *
      output_tokens * model_output_rate
  + tool_charges
  + processing_mode_adjustments
  + regional_processing_adjustments
  + retry_request_costs

The most important dashboard correction is to distinguish “eligible to be cached” from “actually reused.” OpenAI’s prompt-caching guide says a session does not guarantee a cache hit and that cache location is machine- and region-dependent. It also says cache keys do not guarantee a hit or pin a request to a machine. Therefore, dashboards should label actual reuse only from usage fields and should label expected reuse, predicted reuse, breakpoint coverage, and diagnostic classification as separate advisory signals.

Attribute write, read, miss, output, and tool spend to the right tenant and owner

Tenant accounting is not just a chargeback exercise; it is also a privacy and probe-resistance control. OpenAI’s documentation states that caches are not shared across organizations or regional processing boundaries, but within your own application you still need to decide whether a stable prefix belongs to a workspace, customer tenant, environment, project, department, or individual workflow. The safest default is to attribute any cache write to the tenant or service owner that caused it, and to attribute cached reads to the tenant or service owner that benefited from the reuse, without creating cross-tenant reuse incentives for sensitive or customer-specific content.

Use a two-ledger model when shared platform components and tenant-specific work coexist. The first ledger is the request ledger, which records the complete request-level evidence. The second ledger is the allocation ledger, which converts request evidence into billing, showback, or cost-center entries. This separation lets security teams preserve raw forensic detail while finance teams receive stable categories such as “platform reusable instruction,” “tenant reusable context,” “tenant variable input,” “output,” “tools,” “retry,” and “prewarm.”

Cost component	Recommended attribution	Evidence to retain	Approval trigger
Platform instruction cache write	Platform AI infrastructure owner, unless a tenant-specific customization caused the write.	Prompt version, breakpoint ID, model, region, service tier, tool manifest hash, schema hash, and approving change ticket.	Any change to shared instruction content, cache mode, region, or retention assumption.
Tenant prefix cache write	Tenant, workspace, or project that supplied the reusable context.	Tenant ID, cache-key namespace, data-classification label, redacted content category, and write timestamp.	Any new sensitive-data class, cross-workspace reuse proposal, or prewarm plan.
Cached read	Benefiting tenant or workflow, with aggregation where user-level reporting could reveal probe signals.	Cached-token count, prompt-cache key, comparison cohort, model, and service tier.	Unexpected read spikes, reads from a new region, or reads after a policy boundary change.
Uncached miss	Requesting tenant or workflow, plus root-cause category after diagnostic review.	Uncached-token count, diagnostic type if used, first classified reason, deployment version, and prompt diff category.	Miss rate threshold breach, repeated schema/tool drift, or unexplained service-tier changes.
Output and reasoning spend	Requesting tenant or workflow, because caching does not discount output generation.	Output token count, visible-output length, reasoning effort or mode when applicable, incomplete status, and retry decision.	Unexpected verbosity changes, reasoning effort escalation, or retries without human-approved policy.
Tool spend	Requesting tenant or workflow, unless a platform tool call was caused by shared orchestration.	Tool name, allowed-tools policy, tool-call count, tool result size, tool-side cost record, and side-effect classification.	New tool, broader tool authority, external action, payment, booking, publication, destructive operation, or permission change.

Do not present prompt-cache savings as a discount owed to a tenant unless your contract, internal chargeback policy, and evidence support that treatment. In many platforms, cached reads reduce infrastructure cost for the operator while tenants are billed for a managed service or a task outcome. In other platforms, tenants receive direct pass-through billing or transparent token-based showback. Either model can be reasonable, but hiding cache writes, reads, misses, or prewarm costs from stakeholders invites disputes during incident review.

For privacy-conscious customers, provide aggregated cache metrics that do not reveal whether a specific user’s prompt matched another specific user’s prompt. A safe tenant report can show monthly cached-read tokens, write tokens, uncached input, output, tool charges, and cost variance by application route. A riskier report would expose per-user timing of cache hits and misses for a shared prefix, because that timing could support probing of whether another user or workflow recently submitted similar content. The operational rule is simple: the more sensitive the content and the smaller the cohort, the more you should aggregate, delay, or suppress cache-hit detail.

This OpenAI Agents API security and data guide explains network controls, sandbox expiry, residency, artifact retention, and the documented absence of Zero Data Retention support, giving cache-governance teams a provider-specific retention reference. The OpenAI Agents API Security and Data Guide: Network Allowlists, Sandbox Expiry, US Residency, No ZDR, and Artifact Retention article is a focused companion for Zero Data Retention because it is an OpenAI-specific data-retention guide and therefore a stronger fit than the draft Anthropic safeguard article.

Build baseline cohorts before alerting on cache anomalies

Anomaly detection fails when it compares unlike workloads. Cache metrics should be baselined by route, tenant class, model, service tier, region, prompt version, tool manifest, schema, reasoning effort, verbosity setting, compaction policy, cache mode, and long-context band. A GPT-6 Luna batch summarization route with no tools should not be compared with a GPT-6 Sol agentic coding route using hosted tools, structured outputs, and high reasoning effort. The first route’s economics may be dominated by repeated stable instructions; the second may be dominated by output, reasoning tokens, tool results, and variable source context.

Use baseline cohorts that reflect production reality rather than vendor benchmark categories. OpenAI’s launch and model-reference pages provide useful model-positioning and pricing evidence, but your cache behavior depends on your rendered prompts, routing, tool order, schema stability, user distribution, region, service tier, and conversation-management code. A baseline cohort should contain requests that are intended to share the same prefix architecture and cost policy. If a route intentionally changed from implicit caching to explicit-only mode, it needs a new baseline because no-breakpoint behavior differs under explicit-only operation.

Baseline dimension	Examples of cohort values	Reason to separate
Model	gpt-6-luna, gpt-6-sol, gpt-6-astra	Prices, effort options, knowledge cutoffs, and effective behavior differ by model; Astra also does not list none effort on the cited model page.
Service tier or processing mode	Standard, Batch, Flex, Fast, or another supported setting available to the account and model.	OpenAI’s model pages document materially different pricing for Batch/Flex and Fast mode, and diagnostics can classify service tier as a miss reason.
Region and data residency	Default processing region, regional processing where available, EU data residency where available under documented constraints.	OpenAI states caches are not shared across regional processing boundaries; regional processing can also add premiums where available.
Prompt version and breakpoint plan	Instruction version, explicit breakpoint IDs, append-only policy version.	Earlier-input changes and breakpoint movement can create misses even when the task looks identical to the user.
Tools and schema	Tool definition hash, tool ordering hash, structured-output schema hash, text format setting.	OpenAI documents tools, tool ordering, structured-output schema, and text format as cache-sensitive reasons.
Reasoning and verbosity	Reasoning effort, mode where applicable, verbosity setting, appended configuration-update policy.	Top-level reasoning changes and verbosity can affect reuse; GPT-6 supports an appended configuration_update for changing effort without rewriting the earlier prefix when supported.
Context band	Under 1,024 visible tokens, 1,024–272,000 input tokens, above 272,000 input tokens.	OpenAI documents a 1,024-token minimum cacheable visible prefix for GPT-5.6 and later, and long-context multipliers above 272,000 input tokens.

Set separate baselines for cold-start windows and steady-state windows. A newly deployed prompt version may show elevated write spend because it has not yet accumulated eligible reuse. That can be acceptable if a documented rollout plan predicts the write burst and the read rate improves inside the 30-minute eligibility window. Conversely, a route that has been stable for days but suddenly writes new prefixes every minute may be experiencing prompt drift, tool-order nondeterminism, schema churn, compaction, or unapproved personalization in the cacheable prefix.

Store baseline snapshots with the deployment artifact. A cache anomaly investigation should be able to answer: what was the expected cached-token ratio for this route, what prompt version established that expectation, what model and service tier were used, what tool manifest and schema were active, what tenant isolation rule applied, and whether long-context multipliers were expected. Without that snapshot, teams often debate whether a cost spike is a caching failure, a successful migration to a more expensive model, a tool-side regression, or legitimate growth in user traffic.

Define anomaly thresholds that catch cost risk without punishing legitimate change

Thresholds should combine absolute spend, token mix, and configuration drift. A small route can tolerate high percentage variance without material impact; a high-volume route can create a finance incident with a small percentage change. Use multi-level alerts: informational alerts for cache-efficiency drift, warning alerts for unexpected write or miss growth, and incident alerts for material spend, tenant-boundary concerns, regional-policy violations, or unapproved tool authority changes. Human approval is required before changing tenant boundaries, retention policy assumptions, regional processing, sensitive-data classification, billing attribution, or incident response posture.

Recommended thresholds should be tuned locally, but the following starting rules are practical for GPT-6 cache FinOps. Treat them as policy examples, not OpenAI guidance. First, alert when a route’s cached-read-token share drops materially below its trailing baseline for two consecutive comparable windows. Second, alert when cache-write tokens exceed the expected rollout envelope after a deployment freeze. Third, alert when uncached input tokens grow while request count and task mix remain stable. Fourth, alert when output or reasoning tokens increase enough to erase input-cache savings. Fifth, alert immediately when a route crosses the 272,000-input-token boundary unexpectedly, because OpenAI documents multipliers that apply to the full request above that threshold.

Anomaly	Primary signal	Likely causes to test	Immediate response
Read collapse	Actual cached tokens fall sharply for the same route and comparable traffic.	Model change, service-tier change, tool/schema drift, reasoning setting change, compaction, earlier-input rewrite, region change, or cache-key namespace change.	Freeze prompt and tool changes, capture request evidence, and run diagnostics against recent comparable responses where supported.
Write storm	Cache-write tokens or write count spike without a planned rollout or prewarm approval.	Dynamic content inserted before breakpoint, nondeterministic tool ordering, per-user instructions in the shared prefix, or explicit breakpoints placed too late.	Disable unapproved prewarming, revert prompt rendering if safe, and require approval before changing cache policy.
Hit rate improves but cost rises	Cached-token share increases while total request cost or task cost also increases.	Longer prompts, high output verbosity, reasoning effort increase, pro mode, tools, retries, Fast mode, regional premium, or long-context multiplier.	Break down cost by input, write, read, output, tool, region, processing mode, and retry path before claiming caching failed.
Long-context cliff	Input tokens exceed 272,000 for a route that usually stays below the threshold.	Retrieval over-inclusion, conversation compaction failure, document duplication, tool-result bloat, or missing truncation policy.	Stop automatic expansion, inspect retrieval and context-management changes, and require product approval for sustained long-context operation.
Tenant cost inversion	One tenant receives read benefits while another tenant or platform owner absorbs unexpected writes.	Misattributed shared prefix, prewarm billed to wrong cost center, or ambiguous platform-versus-tenant instruction ownership.	Correct allocation rules prospectively, preserve evidence, and avoid retroactive billing changes without finance and legal review.
Probe-risk pattern	User-visible timing or cost responses correlate too directly with cache hits for small cohorts.	Overexposed per-user cache metrics, shared sensitive prefixes, or support tooling that reveals hit/miss timing.	Aggregate or suppress fine-grained signals, isolate cache accounting, and review whether the prefix should be cacheable for that cohort.

Do not auto-remediate every anomaly by stripping tools, lowering reasoning effort, or changing models. Those actions can alter safety properties, task quality, permissions, and audit assumptions. For example, moving a coding workflow from Sol to Luna solely because write spend increased could reduce cost while increasing review burden or failure risk for complex agentic work. Conversely, moving to Astra may improve fit for hard work but increase input, cached-input, write, and output rates. Route changes require evaluation evidence and rollback planning, not only FinOps pressure.

Alert on missing evidence as aggressively as you alert on high spend. If usage records stop including cached-token detail, if deployment artifacts lose prompt-version hashes, if tool manifests are not versioned, or if tenant identifiers are absent from the request ledger, finance can no longer prove whether a cost change is valid. Missing observability should block broad rollout in the same way a failing latency or accuracy metric would block rollout.

Use diagnostics as evidence, not as the source of truth

OpenAI’s prompt-cache diagnostics are available in the Responses API for supported GPT-5.6-and-later models. A request can compare against a recent completed response by setting prompt_cache_options.comparison_response_id. OpenAI states this asks for diagnostic metadata; it does not load the earlier conversation, does not change caching behavior, and does not block or fail the model request. The diagnostics feature has no additional diagnostic fee according to OpenAI, but any extra baseline, comparison, or retry model requests remain billable.

The diagnostics result types documented by OpenAI are cache_hit, cache_miss, comparison_response_not_found, and unavailable. A miss can return a reason and token estimates, including categories such as model, cache key, service tier, tools, text format, reasoning effort, verbosity, compaction, and input changes. Diagnostics are best effort and return the first classified reason. This matters operationally because the first reason is not necessarily the only reason. A route can have both a tool-order change and an earlier-input rewrite; the diagnostic response may classify only the first detected reason.

This guide covers production-grade prompt engineering practices for frontier AI APIs, including layered prompt architecture, caching, and agentic task optimization. The The 2026 ChatGPT Prompt Engineering Best Practices Guide article is a focused companion for Prompt Cache Diagnostics because among the candidates, it most directly connects prompt-system design with caching practices, which supports diagnostics for cache hit behavior and prompt structure.

Use diagnostics in a controlled workflow. Select a recent completed response from the same intended baseline cohort, then compare the current request against that response. Do not compare unrelated tenants, regions, models, tool policies, schemas, or prompt versions. If the comparison record has expired or cannot be found, record comparison_response_not_found as evidence and fall back to request-ledger diffing. If diagnostics are unavailable, do not retry indefinitely; preserve the unavailable status, examine local diffs, and avoid creating extra billable traffic without an approved investigation plan.

{
  "purpose": "Recommended diagnostic evidence record; not an OpenAI response schema contract",
  "current_response_id": "stored internally by your application",
  "comparison_response_id": "stored internally by your application",
  "route_id": "support_summarization_v4",
  "tenant_namespace": "workspace-level namespace or approved aggregate",
  "model": "gpt-6-sol",
  "service_tier": "recorded service tier",
  "region_policy": "recorded processing boundary",
  "prompt_version": "sha256-or-version-id",
  "tool_manifest_hash": "sha256-or-version-id",
  "schema_hash": "sha256-or-version-id",
  "reasoning_effort": "recorded model-specific setting",
  "verbosity": "recorded setting if used",
  "context_management_policy": "append-only-v3",
  "actual_cached_tokens": 0,
  "diagnostic_type": "cache_miss",
  "diagnostic_first_reason": "tools",
  "investigator_note": "Tool ordering changed between deployment 2026-09-22.3 and 2026-09-22.4"
}

Never treat diagnostics as permission to expose raw prompts, personal records, customer data, secrets, or privileged material in logs. OpenAI states the diagnostic feature is compatible with Zero Data Retention and stores configuration metadata, token-count estimates, and hashes rather than raw prompts or outputs for diagnostics. That statement does not review your application logs, observability vendor, data warehouse, support exports, browser telemetry, or incident ticket attachments. Security teams should separately approve what your systems store, who can query it, how long it is retained, and whether it crosses tenant, region, or contractual boundaries.

Diagnose the common miss classes in a repeatable order

Start every cache-miss investigation by preserving the current request record before changing anything. Capture model ID, endpoint surface, service tier, processing mode, region, tenant namespace, prompt-cache key, cache mode, explicit breakpoints, rendered prompt hash segments, tool definition hash, tool ordering hash, structured-output schema hash, text format, reasoning effort, verbosity, context-management state, input-token count, cached-token count, output-token count, tool calls, retry count, and response status. If the request may involve a billing dispute, tenant boundary, sensitive-data classification, or incident response, preserve evidence under the organization’s approved legal and security process rather than letting engineers overwrite it during debugging.

Then test the miss classes in an order that separates configuration drift from content drift. The order below is a recommended workflow for operations teams. It aligns with OpenAI’s documented diagnostic reason categories, but it is not a guarantee that OpenAI diagnostics will classify the same way in every case, because diagnostics are best effort and return the first classified reason.

1. Model check: Confirm the request used the intended model ID, such as gpt-6-luna, gpt-6-sol, or gpt-6-astra. A fallback, canary, emergency route, or workspace default change can move traffic without a prompt change. Model switches can affect cache compatibility and price.
2. Service-tier and processing-mode check: Compare Standard, Batch, Flex, Fast, regional processing, and any account-specific tier settings recorded for the baseline. OpenAI diagnostics include service tier as a possible miss reason, and OpenAI model pages document pricing differences for several processing modes.
3. Regional boundary check: Confirm the processing region and data-residency policy. OpenAI’s prompt-caching guide states caches are not shared across organizations or regional processing boundaries, so a region change can make an otherwise stable prefix miss.
4. Cache-key namespace check: Verify that the prompt_cache_key, if used, remained in the intended accounting or isolation namespace. OpenAI says the key is not required for GPT-5.6-and-later routing and does not guarantee a hit, but changing it can affect your own accounting and isolation behavior.
5. Tool-definition check: Compare tool definitions and ordering. OpenAI’s caching guidance says tool definitions, schemas, and ordering should remain stable, and suggests using allowed_tools or tool_choice: none rather than removing definitions when possible.
6. Schema and text-format check: Compare structured-output schema, response format, and text format settings. A small schema reordering or generated enum change can invalidate a prefix even when the visible product workflow did not change.
7. Reasoning-effort check: Compare effort and mode settings. OpenAI says top-level reasoning changes can affect cache reuse, while GPT-6 can change reasoning effort through an appended configuration_update without rewriting the earlier cached prefix when supported. Validate model-specific effort support before applying a shared configuration.
8. Verbosity check: Compare verbosity settings and prompt instructions that control response length. Verbosity can be a diagnostic reason and can also increase output cost enough to mask cache savings.
9. Compaction check: Inspect whether conversation history was summarized, trimmed, reordered, or rewritten before the cacheable prefix. OpenAI documents compaction as a possible miss reason; compaction may be necessary for quality or context limits, but it changes the rendered prefix.
10. Earlier-input check: Diff the rendered prefix before the breakpoint or before the minimum cacheable region. User personalization, timestamps, randomized IDs, experiment labels, retrieved snippets, or dynamic policy text inserted too early can create misses.
11. Long-context check: Determine whether the request crossed 272,000 input tokens. Above that threshold, OpenAI documents 2x input and cache rates and 1.5x output rates for the full request, so a cost anomaly may be a multiplier event rather than a pure cache miss.
12. Actual-usage check: Reconcile diagnostic output with usage.input_tokens_details.cached_tokens, output tokens, tool calls, and retry records. Actual usage is the billing-adjacent evidence; diagnostics explain likely causes but do not replace usage accounting.

Model drift is often caused by fallback code rather than deliberate migration. For example, a route may attempt GPT-6 Sol and fall back to GPT-6 Luna during a transient error, or an internal experiment may route a percentage of traffic to Astra. That can be valid if approved and measured, but it must be visible in the cache ledger because the models have different token rates and may support different reasoning settings. Validate model-specific settings rather than copying one effort configuration across the GPT-6 family; according to the cited model pages, Sol and Luna list none effort while Astra does not.

Tool drift is a common hidden miss source because product teams often treat tool availability as an authorization layer instead of part of the rendered prefix. OpenAI’s caching guidance recommends stable tool definitions and changing callability with controls such as allowed_tools or tool_choice: none where possible. The FinOps reason is that removing, reordering, or regenerating tool definitions can reduce reuse. The security reason is that tool authority changes are consequential and require review; do not broaden tool access merely to improve cache reuse.

Schema drift can be introduced by build systems, code generation, or feature flags. If a structured-output schema includes generated ordering, optional descriptions, tenant-specific enum labels, or timestamps, the rendered prefix may change even when the product team believes the schema is stable. Store a canonical schema hash with each request and include a human-readable schema version in deployment notes. If a schema change is required for correctness, accept the cache miss and reset the baseline rather than preserving a stale schema for cost reasons.

Reasoning and verbosity drift can hide inside application defaults. OpenAI’s reasoning documentation says supported effort values are model-dependent, that GPT-6 Sol and Luna default to medium effort, and that reasoning tokens are billed as output tokens. The cache dashboard should therefore show reasoning effort and output tokens next to cached-token metrics. If cache reads improve but reasoning effort also increases, the task may become more expensive while the cache layer is working as designed.

Compaction drift deserves special attention in long-running conversations. A compactor that summarizes earlier turns may improve context management, remove sensitive material, or keep a request under context limits. It can also rewrite the prefix and reduce cache reuse. The correct response is not to disable compaction universally; it is to define which turns are stable, which turns may be compacted, how summaries are versioned, and whether the compaction boundary sits after the reusable prefix whenever that is semantically safe.

Turn cost anomalies into defensible evidence packages

A defensible anomaly package should let finance, engineering, security, and product agree on what happened without exposing unnecessary confidential content. The package should include a timeline, affected tenants or cohorts, request counts, input-token totals, actual cached-token totals, cache-write-token totals, uncached-token totals, output-token totals, tool charges, long-context multiplier events, service-tier distribution, region distribution, retries, diagnostic results, and deployment changes. Use hashes, version IDs, category labels, and redacted excerpts rather than raw prompts wherever possible.

Classify the anomaly before prescribing a fix. A cache-efficiency anomaly means the cache layer reused fewer tokens than expected. A cost anomaly means total spend increased, which may be caused by output, tools, mode, region, long context, or retries even when cache reuse improved. A governance anomaly means a tenant boundary, data-classification rule, regional policy, or approval rule may have been violated. An evidence anomaly means records are missing or inconsistent. These categories require different owners and different approval paths.

Evidence item	Minimum content	Do not include
Timeline	Deployment times, prompt-version changes, prewarm windows, traffic shifts, diagnostic runs, and rollback actions.	Speculative blame or unverified root-cause claims.
Token and cost ledger	Uncached input, cache writes, cached reads, output, tools, retries, service tier, region, and long-context flags by cohort.	Blended averages that make write/read/output paths impossible to audit.
Configuration diff	Model, cache mode, breakpoint plan, cache key namespace, tools, schema, effort, verbosity, compaction, and earlier-input category changes.	Passwords, tokens, private keys, privileged source excerpts, personal records, or unnecessary customer content.
Diagnostics summary	Diagnostic type, first classified reason, comparison response age, unavailable/not-found cases, and actual cached-token reconciliation.	Claims that diagnostics prove the only cause or that a diagnostic hit means the whole prompt was cached.
Decision record	Approved mitigation, owner, risk acceptance, rollback criteria, customer-communication decision, and follow-up monitoring.	Unapproved commitments, legal conclusions, or promises of future cost savings.

When the anomaly affects customer billing or tenant allocation, involve finance and legal-operations stakeholders before changing historical charges. Engineering can provide evidence about token paths and configuration drift, but billing corrections, customer credits, contractual interpretation, and public statements require authorized review. The same conservative rule applies to security incidents: do not declare that no data risk exists merely because a cache diagnostic stores hashes rather than raw prompts. Review your application logs, support tooling, regional boundaries, and access records before making a conclusion.

Rollback must restore the working bundle, not just the model ID. If the cause was a tool-order change, rolling back only from Astra to Sol or Sol to Luna will not restore cache reuse. If the cause was compaction or earlier-input rewriting, a model rollback may leave the rendered prefix unstable. A complete rollback bundle includes model selection, prompt/cache policy, breakpoint placement, cache-key namespace, tool definitions and order, schema version, reasoning and verbosity settings, context-management behavior, region policy, and monitoring thresholds.

Operational rule: prompt-cache monitoring is successful when a reviewer can distinguish a planned cache-write investment from an accidental write storm, a true cache miss from a long-context multiplier event, and a tenant-specific cost from a shared-platform cost without inspecting raw sensitive prompts.

Incident runbook: contain spend, preserve evidence, and protect controls

A prompt-cache incident is any event where cache behavior, cache accounting, or cache governance materially diverges from the approved design. Treat unexpected spend, tenant attribution errors, cross-user probe concerns, regional-policy drift, sensitive-prefix exposure, unexplained cache misses, and misleading hit-rate reporting as incident classes, not as ordinary dashboard noise. The first decision rule is simple: never hide costs, merge tenants, weaken data controls, broaden regional routing, or suppress evidence to make cache metrics look better.

OpenAI’s prompt-caching documentation says supported GPT-5.6-and-later models can write cache entries at 1.25 times the uncached input rate and read eligible cached input at 0.1 times that rate. That makes write/read classification a financial control, not just a performance optimization. If a dashboard reports a high hit rate while spend increases, the team must inspect uncached input, cache writes, cached reads, output and reasoning tokens, tool charges, retries, processing mode, regional premiums, and long-context multipliers separately before declaring the system healthy.

The incident commander should freeze the situation quickly without destroying forensic value. Pause optional prewarming, stop nonessential experiments, disable new prompt-template rollouts, and suspend routing changes that were not already approved. Do not delete request records, token-accounting rows, diagnostic comparison identifiers, prompt-version metadata, or tenant billing mappings unless counsel or the organization’s retention policy requires a specific action. Evidence preservation is especially important because OpenAI states that cache diagnostics are best effort, may expire after a short period, and report the first classified reason rather than every contributing cause.

Operational rule: optimize for truthful attribution and safe rollback before optimizing for hit rate. A lower cache-miss rate is not a valid reason to reuse tenant-specific prefixes across tenants, bypass regional processing rules, remove sensitive-data classification, or make external actions without authorized human approval.

Incident classification table

Incident class	Typical signal	Immediate containment	Evidence to preserve	Human approval required before
Unexpected spend	Daily or hourly cost exceeds baseline after a cache, model, prompt, routing, or tool change.	Pause optional prewarming, freeze prompt-template releases, cap noncritical batch volume, and route only approved workloads.	Usage rows by tenant, model, processing mode, uncached input, cache write, cached read, output, tool calls, retries, and request size.	Changing billing attribution, raising budgets, rerouting regulated workloads, or restarting prewarming.
Tenant-accounting defect	Writes or reads are charged to the wrong tenant, product, workspace, region, or cost center.	Stop shared chargeback publication and move affected tenants to conservative direct attribution until mapping is corrected.	Cache key policy, tenant mapping table, request IDs, prompt versions, workspace identifiers, and allocation formulas.	Issuing credits, invoices, customer reports, or changing tenant boundaries.
Probe concern	A user, tenant, or test harness may infer another user’s cache state from timing, cost, diagnostics, or hit-rate feedback.	Reduce exposed cache telemetry, isolate accounting views, disable user-visible fine-grained hit indicators, and escalate to security review.	Telemetry exposures, UI/API fields shown to users, diagnostic calls, timing logs, and tenant-isolation configuration.	Restoring user-visible cache diagnostics or changing isolation strategy.
Regional-policy drift	Requests appear in the wrong processing region, use an unapproved processing mode, or cross a policy boundary.	Stop affected routes, revert to the last approved regional configuration, and block fallback paths that ignore region constraints.	Region flags, service tier, model ID, workspace policy, cache key namespace, deployment config, and change approvals.	Re-enabling cross-region failover, changing residency policy, or resuming affected workloads.
Sensitive-prefix exposure	Secrets, personal data, regulated records, confidential code, or tenant-specific content entered a stable cacheable prefix.	Stop the template or workflow, rotate exposed secrets if any, disable affected prewarming, and move to a sanitized prompt version.	Redacted prompt versions, classification decisions, request metadata, affected tenants, and remediation timestamps.	Restarting the workflow, notifying customers, or changing retention and logging policy.
Cache-miss regression	Cached-token share drops after a model, tools, schema, reasoning, verbosity, compaction, or prompt-order change.	Freeze the last known-good configuration and compare against recent completed responses using diagnostics where available.	Prompt diffs, tool definitions and ordering, structured-output schema, reasoning settings, verbosity, compaction events, and diagnostics.	Rolling forward with higher cost, modifying stable prefixes, or changing user-visible pricing assumptions.
Misleading hit-rate reporting	Dashboard labels diagnostic hits as full-cache reuse or excludes writes, retries, long-context multipliers, or tool costs.	Remove or mark the report as unreliable, freeze external reporting, and publish corrected definitions internally.	Dashboard query versions, metric definitions, cached-token fields, excluded charges, and distribution lists.	Customer communication, executive reporting, or billing-policy updates.

This playbook covers AI misuse incident response for stolen API keys and agentic cyber operations, including detection, containment, and evidence handling. The AI Misuse Incident Response Playbook: Stolen API Keys, Agentic Cyber Operations, Detection, Containment, and Evidence article is a focused companion for Incident Evidence and Rollback because the current marker emphasizes evidence for anomalies and rollback decisions, and this target provides the closest incident-response and evidence-management context without duplicating earlier selections.

Containment procedure for unexpected spend

Start with a spend freeze that preserves service for approved production use. Disable optional prewarming, halt experimental traffic, pause bulk replays, and stop new prompt-cache breakpoint changes. If the workload has separate priority lanes, keep only the lane required for customer commitments and safety obligations. Do not lower review requirements, remove tool restrictions, or move users to less protected regions merely to reduce token cost.

Next, split the cost path into separate billable components. For GPT-6 Luna, Sol, and Astra, OpenAI’s model references publish distinct prices for uncached input, cached input, cache writes, and output tokens, and they also document that requests above 272,000 input tokens trigger multipliers for the full request. A spend spike can therefore come from larger rendered prefixes, more cache writes, fewer reads, higher output or reasoning token usage, retries after incomplete responses, tool fees, Fast mode, regional processing premiums where available, or a move to a different model.

1. Identify the change window. Record the first timestamp where spend, cache-write volume, cached-token share, request size, retry rate, model mix, or processing mode diverged from the baseline.
2. Freeze mutable inputs. Lock prompt templates, tool definitions, schema versions, reasoning settings, verbosity settings, compaction rules, cache-key construction, and routing policies.
3. Separate new writes from reads. A request can create cache writes, and a write is more expensive than uncached input for that prefix. A single cache write is not automatically a saving; savings depend on later reuse and the rest of the request.
4. Check long-context thresholds. If rendered input crossed the documented threshold above 272,000 input tokens, apply the relevant multiplier to the full request rather than only to the excess portion.
5. Inspect output and reasoning growth. Reasoning tokens are billed as output tokens in supported reasoning workflows, even when not visible. A cache improvement can be financially overwhelmed by longer outputs or higher effort settings.
6. Confirm tool and retry behavior. Tool calls may add separate charges, and retries after errors or incomplete responses can multiply total task cost even if each individual prompt has some cached prefix reuse.

Correction must be explicit and reversible. If a prompt rewrite caused miss regressions, roll back the prompt and cache policy together. If a reasoning setting change caused reuse loss, use only model-supported settings and preserve the cacheable prefix when semantically correct; OpenAI notes that on GPT-6 an appended configuration_update can alter reasoning effort while preserving the earlier prefix, whereas top-level reasoning changes can affect cache reuse. If long-context growth caused a multiplier event, reduce unnecessary context by policy-approved pruning rather than by removing audit-critical instructions or access-control context.

{
  "incident_class": "unexpected_spend",
  "containment_status": "optional_prewarming_paused",
  "change_window_utc": {
    "start": "recorded_timestamp",
    "end": "ongoing_or_recorded_timestamp"
  },
  "required_cost_breakout": [
    "uncached_input_tokens",
    "cache_write_tokens",
    "cached_input_tokens",
    "output_tokens",
    "reasoning_tokens_if_reported",
    "tool_charges_if_applicable",
    "retries",
    "processing_mode",
    "regional_processing",
    "long_context_multiplier"
  ],
  "approval_required_for": [
    "budget_increase",
    "customer_billing_adjustment",
    "regional_policy_change",
    "prewarming_restart",
    "prompt_cache_policy_change"
  ]
}

Tenant-accounting defects and customer correction

Tenant accounting is defective when a write, read, miss, output charge, tool charge, retry, or shared infrastructure allocation is attributed to the wrong tenant or cost owner. Because OpenAI’s documentation says caches are not shared across organizations or regional processing boundaries and cache keys can be used for accounting or isolation without guaranteeing a hit or pinning a request to a machine, your accounting system must not infer cross-tenant reuse merely from a shared prefix or a similar request shape.

Containment begins by stopping external chargeback statements for the affected population. Switch to a conservative interim method that prevents overcharging customers while the defect is investigated. For example, if a shared dashboard incorrectly allocated all cache writes to the first tenant that warmed a prefix, hold those charges in a suspense account until the allocation policy is corrected and approved. Do not silently move the cost to another tenant to preserve margin.

Evidence should include the exact mapping logic used at request time, not only the corrected logic. Preserve request IDs, tenant IDs or redacted tenant aliases, workspace or product identifiers, cache-key components, prompt-version identifiers, region, model ID, service tier, and the accounting query version. If personally identifiable or confidential information appears in logs, preserve the existence and classification of the evidence while restricting access and using redacted extracts for broad incident review.

Customer communication should be plain and bounded. State what was wrong, what period is affected, which cost components were affected, whether invoices or internal showback reports changed, and what correction will occur. Do not disclose another tenant’s usage, cache behavior, prefix content, region, or security posture. If the impact is still being quantified, say that the investigation is ongoing and provide a follow-up schedule rather than speculating.

Recommended customer notice structure

Notice section	Concrete content to include	Content to avoid
Summary	A concise statement that prompt-cache accounting or reporting was inaccurate for a defined period.	Claims that the incident had no impact before evidence supports that conclusion.
Financial impact	The affected billing components, correction method, and whether credits or corrected statements will be issued.	Another tenant’s costs, usage, cache-hit patterns, or prompt contents.
Data controls	Whether the issue was limited to accounting or also involved policy, region, or sensitive-prefix exposure.	Overbroad privacy assurances not supported by the investigation.
Remediation	Rollback, corrected queries, approval gates, and post-incident tests.	Promises of guaranteed future savings or perfect cache-hit rates.

Probe concerns and misleading hit-rate reporting

A probe concern exists when a user or tenant might learn something about another user’s cache state through timing, cost deltas, diagnostic feedback, detailed hit indicators, or aggregate reports with too little aggregation. OpenAI states that prompt caching reduces repeated computation; it does not authorize data sharing, validate content, or replace application controls. The safest design is to expose only the minimum cache telemetry each user needs and to keep fine-grained diagnostic evidence behind authorized operational access.

Containment should reduce observability to potential attackers without destroying the evidence needed by security and FinOps teams. Remove user-visible per-request cache-hit badges if they could reveal another user’s recent activity. Delay, aggregate, or suppress tenant-visible cost components that allow reverse engineering of another tenant’s prefix reuse. Keep internal records of usage.input_tokens_details.cached_tokens, cache writes, and diagnostics, but restrict them to personnel with a documented need to investigate the incident.

Misleading hit-rate reporting often looks like a security or finance incident because it changes behavior. A dashboard that treats a diagnostic cache_hit as proof that the entire prompt was cached can cause teams to ignore uncached tail tokens, output tokens, tool charges, and retries. OpenAI’s diagnostics documentation says a diagnostic hit means no comparison miss was detected, not that the entire current prompt came from cache; actual reuse is measured through cached-token usage fields. Correct the metric label, recalculate historical reports if decisions were based on the flawed metric, and notify stakeholders who relied on it.

Regional-policy drift and sensitive-prefix exposure

Regional-policy drift is a governance incident even when the bill decreases. OpenAI’s prompt-caching documentation says cache location is machine- and region-dependent and that caches are not shared across organizations or regional processing boundaries. The model references also state that regional processing can add a premium where available for Sol and Luna and that EU data residency for Sol and Luna is available only with Standard processing. Do not switch processing modes, regions, or fallback paths to improve cache reuse unless the data-region policy owner approves the change.

Contain regional drift by restoring the last approved region and processing-mode configuration. Block fallback code that drops regional constraints when a cache miss or latency spike occurs. Preserve deployment configuration, routing policy, workspace policy, service tier, cache-key namespace, and the exact request metadata showing the drift. If affected requests included regulated, confidential, or customer-controlled content, involve legal, privacy, security, and the business owner before any external communication.

Sensitive-prefix exposure requires a separate track. A stable prefix can include developer messages, tool definitions, conversation history, text, images, documents, and supported audio according to OpenAI’s prompt-caching guide. That breadth is useful for reuse but dangerous when secret-bearing or tenant-specific content is accidentally placed into a reusable prefix. Stop the affected workflow, revoke or rotate exposed credentials if any credential-like material was included, replace the prefix with a sanitized version, and review whether application logs, analytics pipelines, or diagnostic exports captured raw content outside the cache system.

Zero Data Retention compatibility for diagnostics must not be overstated. OpenAI’s diagnostics documentation says the diagnostic feature stores configuration metadata, token-count estimates, and hashes rather than raw prompts or outputs and is compatible with Zero Data Retention. That statement does not automatically cover your application logs, data warehouse, tracing system, support tooling, prompt registry, or third-party observability platform. Review each storage path separately before declaring the incident contained.

Cache-miss investigation and rollback bundle

Cache misses should be investigated in an order that matches documented compatibility requirements. OpenAI lists model, cache key, service tier, tools, text format, reasoning effort, verbosity, compaction, and input changes as diagnostic miss reasons. A repeatable order prevents teams from rewriting prompts blindly and creating more misses, more writes, or semantic regressions.

1. Confirm model identity. Compare the current model ID with the baseline. Do not assume that Sol, Luna, and Astra share all settings just because they share a documented context-window size.
2. Check service tier and processing mode. Batch, Flex, Fast mode, Standard processing, and regional processing can change effective cost and may affect operational comparability.
3. Compare cache key construction. If optional cache keys are used for accounting or isolation, confirm that tenant, region, workspace, and purpose components did not change unexpectedly.
4. Diff tool definitions and ordering. OpenAI recommends keeping tool definitions, schemas, and ordering stable where possible; use allowed_tools or tool_choice: none instead of removing definitions when appropriate.
5. Diff structured-output schema and text format. Even small schema changes can alter the rendered prefix and reduce reuse.
6. Inspect reasoning and verbosity settings. Validate model-specific supported values. Astra does not list none reasoning effort in the cited model reference, while Sol and Luna do; fallback code must not copy invalid settings across the family.
7. Review compaction and conversation rewriting. Compaction or rewriting earlier turns can reduce reuse because caching depends on an unchanged rendered prefix.
8. Inspect earlier input changes. A new timestamp, tenant string, request-specific instruction, or generated summary placed too early can break prefix stability.

The rollback bundle must restore more than the model ID. It must restore the prompt template, breakpoint policy, cache mode, tool definitions, tool ordering, schema, reasoning setting, verbosity, state-handling behavior, compaction policy, cache-key construction, regional processing rule, billing attribution logic, and dashboard definitions that were validated together. A partial rollback can leave incompatible assumptions in place and produce a second incident that is harder to explain.

{
  "rollback_bundle": {
    "model_id": "last_approved_model",
    "prompt_template_version": "last_approved_prompt",
    "prompt_cache_mode": "last_approved_mode",
    "breakpoint_policy": "last_approved_breakpoints",
    "tool_definition_version": "last_approved_tools",
    "tool_order_hash": "recorded_hash",
    "structured_output_schema": "last_approved_schema",
    "reasoning_configuration": "last_approved_model_specific_setting",
    "verbosity": "last_approved_value",
    "compaction_policy": "last_approved_policy",
    "cache_key_policy": "last_approved_policy",
    "regional_processing_policy": "last_approved_region_policy",
    "billing_attribution_query": "last_approved_query",
    "dashboard_metric_definitions": "last_approved_definitions"
  },
  "rollback_requires_human_approval": true
}

Post-incident tests before returning to normal operation

Return to normal only after the corrected system passes tests that cover both cost and governance. A prompt-cache fix that restores cached-token share but drops tenant isolation, region enforcement, or sensitive-data classification is not a valid fix. Use representative production-like data that is redacted, synthetic, public, or organization-approved; do not place secrets, private keys, regulated records, or unnecessary personal information into test prompts.

Test	Pass condition	Failure response
Cost reconstruction	Finance can reproduce total task cost from uncached input, writes, reads, output, tools, retries, processing mode, regional premium, and long-context multiplier.	Keep external reporting frozen and correct accounting queries.
Tenant isolation	Requests are attributed to the correct tenant, region, workspace, and cost owner; no cross-tenant prefix sharing is introduced.	Revert cache-key or routing change and escalate to security and finance.
Probe resistance	User-visible telemetry cannot reveal another tenant’s recent cache state through fine-grained hit, timing, or cost indicators.	Aggregate, delay, or remove exposed telemetry and repeat security review.
Regional enforcement	All affected workload classes remain within approved organization and regional processing boundaries.	Block route and require policy-owner approval before retry.
Sensitive-prefix classification	Stable prefixes contain only approved reusable instructions, schemas, and data classes; tenant-specific or secret-bearing material is excluded unless explicitly approved for that tenant and purpose.	Stop workflow, sanitize prefix, and review logging exposure.
Diagnostics interpretation	Reports distinguish diagnostic hit/miss from actual cached-token usage and label best-effort diagnostics accurately.	Correct dashboard definitions and notify prior report recipients.
Rollback rehearsal	The team can restore the full rollback bundle in a controlled environment without losing audit evidence.	Keep canary disabled and update runbooks before production restoration.

Use staged reactivation. First, replay a small approved holdout set in a non-side-effecting environment. Next, run a canary for a low-risk tenant or internal workload with explicit cost and policy monitors. Finally, return broader production traffic only after the incident commander, finance owner, security owner, and affected business owner approve the evidence package. External messages, invoice corrections, policy changes, publication, contractual commitments, and customer-facing claims require authorized human approval.

RACI for prompt-cache FinOps incidents

A RACI matrix prevents ambiguous ownership during a cache incident. The incident commander coordinates decisions, but finance owns cost truth, security owns probe and exposure analysis, privacy or legal owns regulated data and notification review, platform engineering owns rollback implementation, and product or customer success owns approved customer communication. Smaller teams can combine roles, but they should not combine approval and implementation in a way that eliminates independent review for consequential decisions.

Activity	Responsible	Accountable	Consulted	Informed
Declare incident and freeze changes	Incident commander	Engineering or operations lead	Finance, security, product owner	Support, affected service owners
Preserve request, usage, and diagnostic evidence	Platform engineering	Security or compliance owner	Legal/privacy, finance	Incident channel participants
Reconstruct cost and attribution	FinOps analyst	Finance owner	Platform engineering, product analytics	Executive sponsor if material
Assess probe or sensitive-prefix risk	Security team	Security owner	Privacy/legal, platform engineering	Customer-success lead if customer impact is possible
Approve rollback bundle	Platform engineering	Engineering lead	Finance, security, product owner	Support and operations
Approve customer communication or billing correction	Customer-success or account team	Business owner	Finance, legal/privacy, security	Support and executive sponsor as appropriate
Close incident and accept residual risk	Incident commander	Accountable service owner	Finance, security, privacy/legal, product	Stakeholders listed in the incident record

Audit checklist for cache governance and anomaly evidence

The audit checklist should be completed before closing the incident and again after the next billing cycle if the incident affected customer charges, internal showback, or capacity planning. The goal is not to prove that prompt caching is perfect; the goal is to prove that the organization can detect, explain, correct, and prevent material defects without weakening privacy, security, or regional controls.

• Scope: The incident record identifies affected models, tenants, workspaces, regions, processing modes, prompt versions, cache modes, tool configurations, and time windows.
• Source of truth: Cost reconstruction uses billable token paths and usage fields, not only dashboard hit-rate percentages or best-effort diagnostics.
• Write/read accounting: Cache writes, cached reads, uncached input, output and reasoning tokens, tool charges, retries, and long-context multipliers are separated.
• Tenant boundaries: The investigation confirms that accounting, cache-key policy, and routing did not introduce cross-tenant sharing or unauthorized cost allocation.
• Regional boundaries: Requests remained within approved organization and regional processing boundaries, or drift was documented, contained, and reviewed by the proper owner.
• Sensitive data: Stable prefixes were reviewed for secrets, personal data, regulated records, confidential code, customer-controlled content, and unnecessary tenant-specific material.
• Diagnostics: Diagnostic outputs are labeled as best effort, short-lived, and first-reason evidence; reports do not treat diagnostic hits as full-request cache reuse.
• Dashboard definitions: Hit rate, cached-token share, write rate, read rate, miss rate, cost per task, and effective task cost are defined in writing and versioned.
• Approvals: Human approvals are recorded for tenant-boundary changes, retention-policy changes, regional processing changes, sensitive-data classification changes, billing corrections, customer communications, and incident closure.
• Rollback: The rollback restored model selection, prompt/cache policy, tool availability, schema, reasoning and verbosity settings, state handling, region rules, accounting logic, and dashboard definitions together.
• Customer impact: External communications and invoice corrections are accurate, bounded, approved, and do not disclose other tenants’ usage or cache behavior.
• Follow-up tests: Post-incident tests cover cost reconstruction, tenant isolation, probe resistance, regional enforcement, sensitive-prefix classification, diagnostics interpretation, and rollback rehearsal.

Conclusion: prompt-cache savings are only useful when they are explainable and governed

GPT-6 prompt caching can materially reduce repeated input computation when prefixes are stable, eligible, and reused within the documented operating assumptions. OpenAI’s published economics make the basic math clear: writes are more expensive than ordinary input, cached reads are cheaper, and the break-even story depends on actual reuse plus output, tools, retries, processing mode, regional premiums, and long-context multipliers. A cache strategy is therefore a FinOps and governance program, not a single prompt-engineering trick.

The defensible operating model is conservative: classify prefixes before caching them, keep tenant and regional boundaries intact, attribute writes and reads honestly, use diagnostics as supporting evidence rather than billing truth, and treat hit-rate claims with precision. A session does not guarantee a hit, a diagnostic hit does not prove full-request reuse, a cache key does not pin a request to a machine, and prewarming does not validate or authorize the content being warmed.

The safest teams will be the ones that can answer hard questions during an incident: who paid for the write, who benefited from the read, what changed, what evidence remains, what was exposed, which region processed the request, who approved the correction, and how rollback restores the entire validated bundle. If a proposed optimization makes any of those answers less clear, it is not an optimization; it is a governance defect waiting to become a billing, security, privacy, or customer-trust incident.

Cache-governance boundary: The documented 30-minute value is a minimum eligibility window after a write or reuse; it is not a retention ceiling and not a guaranteed physical-retention ceiling. Do not share cacheable prefixes across tenants, even when a shared prefix appears cheaper. A cache key supports routing and accounting but does not authorize cross-tenant reuse. Zero Data Retention compatibility for diagnostics does not cover application logging automatically: application logging, telemetry, support records, and other services require separate review and policy.

Useful Links

• OpenAI: Better prompt caching for GPT-6
• OpenAI API documentation: Prompt caching
• OpenAI API documentation: Prompt cache diagnostics
• OpenAI API model reference: GPT-6 Luna
• OpenAI API model reference: GPT-6 Sol
• OpenAI API model reference: GPT-6 Astra