Implement OpenAI Misalignment Monitoring for Agents: Responses API Coverage, 403 Handling, Safety Webhooks, and State Reconciliation

Implement OpenAI Misalignment Monitoring for Agents: Responses API Coverage, 403 Handling, Safety Webhooks, and State Reconciliation
Implement OpenAI Misalignment Monitoring for Agents: Responses API Coverage, 403 Handling, Safety Webhooks, and State Reconciliation

Coverage boundary: Chat Completions is not covered by this misalignment-monitoring system. A monitoring flag is a concern for review, not proof of wrongdoing.

What you are implementing: a defensive monitoring layer, not a verdict engine

OpenAI’s API misalignment monitoring is a safety check for agentic applications that use the Responses API in consequential contexts, including sensitive-data transfer, sensitive-data access, and destructive changes. The system evaluates whether an agent appears to be interpreting user instructions appropriately, and it can raise a flag when the observed interaction creates a safety concern. The first implementation decision is therefore procedural: treat a flag as a stop-and-review signal, not as proof that the user violated policy, not as conclusive evidence that the model acted maliciously, and not as a complete incident report.

The practical goal of this tutorial is to make your application respond safely when OpenAI’s monitoring system blocks a request or creates a safety alert. A correct integration must stop downstream dispatch, preserve enough request lineage to reconstruct what happened, avoid automatic retry, reconcile any actions that may already have completed, verify and deduplicate webhooks, retrieve alert details with same-project credentials, and route the case to a human operator. Those controls matter because OpenAI’s documentation states that monitoring can miss issues and can also flag legitimate work, which means application-level approvals, least privilege, logs, and human review remain mandatory.

For builders migrating from single-turn completions to tool-using agents, the important boundary is API coverage rather than model branding. OpenAI’s misalignment monitoring guide distinguishes several classes of Responses API traffic and explicitly says Chat Completions requests are not covered by this monitoring system, although other safety checks may still apply. If your production agent still uses Chat Completions for consequential tool orchestration, do not assume that the safety alert and block behavior described in this tutorial will appear there; build your own approval gates and migration plan accordingly.

Coverage matrix: which requests can be blocked, alerted, or excluded

The coverage model is the first artifact your engineering, security, and operations teams should agree on, because it determines what your runtime can rely on during an unsafe continuation. OpenAI’s documentation says Responses API requests using persisted reasoning, WebSockets, or OpenAI compaction are monitored in a way that can identify conversation continuation and block further execution. Other Responses API requests are also monitored and can generate webhook alerts, but they are not automatically stopped by this monitoring system. Chat Completions requests are excluded from this specific monitoring system.

Request or conversation mode Misalignment monitoring coverage described by OpenAI Possible platform response Required application behavior
Responses API with persisted reasoning Monitored with continuation awareness when compatible conversation continuity is present. A request can be blocked, including before streaming begins, and project-level alerts may be created. Preserve response lineage, encrypted reasoning items, tool records, and application state; stop dispatch on a block; route human review.
Responses API over WebSockets Monitored in a way that can identify conversation continuation. Further execution can be blocked when the monitoring system registers a concern. Handle mid-session interruption, prevent additional tool execution, and reconcile any actions already started by the session.
Responses API using OpenAI compaction Monitored with continuation awareness across compacted context. A continuation can be blocked when the monitoring system identifies a concern. Treat compaction artifacts as privileged continuity records; keep append-only lineage and do not rewrite summaries to bypass a stop.
Other Responses API requests Monitored, but without the same automatic stopping behavior described for continuation-aware modes. Project-level webhook alerts can be generated, but requests are not automatically stopped by this monitoring system. Use application-level gates for consequential actions, because webhook delivery is not a real-time kill switch.
Chat Completions requests Not covered by this misalignment monitoring system, according to OpenAI’s API guide. No reliance should be placed on this system’s 403 block or safety alert semantics for Chat Completions. Use separate safeguards, moderation where appropriate, human approval, least privilege, and migration analysis if agentic execution is consequential.

This matrix should become an engineering control document, not a slide. For every agent route in your application, record the API used, whether conversation continuity is persisted, whether a WebSocket session can trigger tools, whether compaction is enabled, what external systems can be affected, and which operator queue owns review. When the coverage answer is “alert only” or “excluded,” the application must supply its own stopping mechanism before sensitive transfer, sensitive access, destructive change, or external publication.

The concern-versus-proof distinction must shape the user experience and the incident workflow

OpenAI’s documentation is explicit that a monitoring flag indicates a need for review; it does not prove a policy violation and does not conclusively establish misalignment. This distinction prevents two opposite implementation errors. The first error is underreaction: treating a block as a transient API failure and retrying until it succeeds. The second error is overreaction: automatically accusing a user, revoking access, deleting resources, or changing credentials without human confirmation and independent evidence.

A safe product response should use neutral language such as “This action requires review before continuing” rather than “The model attempted misconduct” or “The user violated policy.” Internally, the case record should carry a status such as “safety concern pending review,” with fields for OpenAI request IDs, application job IDs, user safety identifier, tool calls, external approvals, webhook event IDs, alert IDs, and operator notes. That structure gives investigators the evidence needed to decide whether the event was legitimate work, a prompt-injection attempt, an instruction-boundary failure, a tool authorization problem, or an ordinary false positive.

OpenAI’s safety best-practices guide also recommends defense in depth: moderation where appropriate, adversarial testing, human review for consequential outputs and code, constrained inputs and outputs, clear limitation communication, user reporting channels, and privacy-preserving safety identifiers. In this tutorial’s architecture, those recommendations become concrete controls: a stable hashed end-user identifier instead of a direct personal identifier where sufficient, pre-execution approval for risky tools, append-only tool logs, and a reviewer view that shows original evidence rather than only a model-generated summary.

Architecture overview: where monitoring fits in an agent runtime

The safest mental model is to place OpenAI’s monitoring between model execution and your application’s action dispatcher, while still assuming that some actions may have occurred before a block or alert is observed. A pre-stream block can prevent a response from beginning, but streaming errors can arrive after some output has already been delivered, and a stopped request does not undo earlier actions. OpenAI’s guide also says there is no general resume mechanism for a stopped conversation, so your state machine must support review, reconciliation, and deliberate restart rather than automatic continuation.

Recommended control flow for consequential agent work:

1. Receive user request.
2. Classify requested operation and required approvals.
3. Create an application job record with a privacy-preserving safety identifier.
4. Send a Responses API request using the intended conversation-continuity mode.
5. Persist response IDs, output items, tool-call intents, and streaming checkpoints.
6. Before each consequential tool action, verify local policy and human approval status.
7. If OpenAI returns code misalignment_policy_violation, stop dispatch immediately.
8. If a safety.alert.created webhook arrives, verify, acknowledge, deduplicate, and retrieve details asynchronously.
9. Reconcile already-completed actions against the job ledger.
10. Route the case to an operator with evidence, not conclusions.
11. Restart only through an approved new workflow, never by blind retry of the blocked continuation.

This design deliberately separates model output from action execution. The model may propose a file transfer, credential rotation, database deletion, repository write, customer email, or payment-related action, but the dispatcher should require deterministic policy checks and, where consequential, human approval. If monitoring raises a concern, the dispatcher’s first responsibility is containment: stop new actions, freeze pending tool intents, preserve evidence, and avoid cleanup steps that could destroy the audit trail.

Human approval is mandatory for external transfers, destructive cleanup, credential changes, irreversible incident actions, and consequential decisions. That requirement is not weakened by a webhook, a successful 403 block, or a model-generated explanation. A reviewer should inspect observable evidence such as user instructions, tool-call arguments, application authorization state, source documents, destination allowlists, and completed action logs; the reviewer should not be asked to rely on private model reasoning or on a summary that cannot be traced back to original records.

Request lineage: the audit trail that makes blocks and alerts actionable

Request lineage is the chain of identifiers, conversation items, tool decisions, and application-side state that lets you reconstruct an agent’s path from user request to model response to external action. In OpenAI’s conversation-state guide, Responses API state can be maintained by replaying complete response output, by using a durable Conversation object, or by using previous_response_id. For stateless reasoning-model requests, OpenAI instructs applications to preserve every item in the response output, including encrypted reasoning items and assistant phase values.

That preservation requirement is operationally important because selective replay can change the task lineage that later requests rely on. Do not manually rewrite, omit, or normalize prior output items in a way that changes what the model sees unless your policy explicitly treats the next call as a new reviewed workflow. If a request is blocked, the review packet should include the exact persisted output items your application used, the parent response reference where applicable, the action queue at the moment of interruption, and any external side effects that were already acknowledged by downstream systems.

A minimal lineage schema should include an application job ID, OpenAI request or response identifiers available to your integration, user-facing session ID, privacy-preserving safety identifier, project ID or project mapping, API family used, conversation-continuity mode, model selection record, tool-call ledger, approval ledger, streaming checkpoint, webhook receipt IDs, safety-alert IDs, and reconciliation status. The schema should be append-only: corrections should be new records that point to earlier records, not edits that overwrite what happened.

Lineage field Why it matters during a safety review Implementation warning
Conversation-continuity mode Shows whether persisted reasoning, WebSockets, compaction, manual replay, a Conversation object, or previous_response_id was used. Do not assume all modes have identical stopping semantics under misalignment monitoring.
Full response output items Allows deterministic replay and review of the observable state your application preserved. For reasoning requests, preserve encrypted reasoning items as opaque records; do not claim they reveal raw chain of thought.
Tool-call ledger Separates proposed actions, approved actions, dispatched actions, completed actions, and failed actions. A stopped model request does not reverse tool actions that already completed.
Webhook receipt record Supports deduplication, retry tolerance, and later proof that the receiver verified and acknowledged an event. The webhook event ID is different from the safety-alert ID and should not be treated as the alert record.
Operator decision record Documents whether the case was dismissed, escalated, rolled back, or restarted through a new approved flow. Do not let the agent approve its own restart after a block.

Persisted reasoning and opaque continuity records

Persisted reasoning can support continuity on compatible models, but OpenAI’s reasoning guide does not describe it as exposing private reasoning text. Reasoning items remain opaque, and compatibility is model-family specific; when switching families, incompatible reasoning can be omitted. For implementers, the safe rule is to preserve opaque reasoning artifacts exactly as returned when the API requires them for continuity, while building review workflows around observable messages, tool calls, sources, approvals, and application logs.

Persisted reasoning also affects coverage because OpenAI’s misalignment monitoring guide says Responses API requests using persisted reasoning can be monitored in a continuation-aware way that can block further execution. That does not mean your application can skip state reconciliation. A block may occur after prior tool calls, and there is no general resume mechanism for a stopped conversation. Your state machine should mark the lineage as interrupted, prevent further dispatch, and require an operator-approved new path if the user’s goal remains legitimate.

WebSockets, compaction continuations, and “other Responses” requests

WebSocket-based agent sessions require special care because tool orchestration can feel continuous to the user while the backend is receiving incremental events. If a safety error or stop signal is observed after partial output, the client must stop presenting the agent as still working, the server must stop dispatching queued tool actions, and the job ledger must record which outputs were already shown. Treat partially streamed content as evidence to preserve, not as an instruction to finish the task through another request.

OpenAI compaction continuations also need strict handling because compaction artifacts can become privileged continuity material for later contexts. If a continuation is blocked, do not edit a compaction summary to remove the apparent trigger and resend the conversation. Instead, preserve the compacted state, record the block, and let a human reviewer decide whether a new task should be created with corrected, policy-compliant context. This protects both safety and auditability: the reviewer can see the exact continuity artifact that shaped the blocked request.

Other Responses API requests occupy a different risk category: OpenAI says they are monitored and can generate webhook alerts, but they are not automatically stopped by this monitoring system. For these routes, webhooks should be treated as asynchronous alerting, not control-plane enforcement. If the request can initiate sensitive-data access, sensitive-data transfer, external publication, destructive mutation, or irreversible business action, your application must impose pre-action gates before the webhook ever arrives.

Project-level safety alerts and webhook boundaries

OpenAI projects can subscribe to safety.alert.created webhook events. The webhook contains an alert ID rather than full details, and retrieval must be performed later with a same-project key that has api.safety.alerts.read. The event ID is not the safety-alert ID, alert reasons can be null including for Zero Data Retention requests, and alert retrieval is not a complete audit history. Your application record must therefore combine OpenAI alert metadata with your own lineage, approvals, tool logs, and operator notes.

Webhook reliability should be engineered according to OpenAI’s webhook guide. Preserve the raw request body for signature verification, verify inbound signatures with the project signing secret, acknowledge valid events quickly with a 2xx response, and move non-trivial processing to a background worker. Failed deliveries are retried with exponential backoff for up to 72 hours, redirects are treated as failures, duplicate events can occur, and the webhook-id header should be used as an idempotency key.

Implementation rule: a webhook can notify your project that a safety alert exists, but it should not be treated as exactly-once delivery, a complete audit record, or an automatic stopping mechanism. Verify first, durably record receipt, acknowledge quickly, deduplicate, retrieve details asynchronously, and escalate to a human review queue.

The request_paused=true field requires especially careful interpretation. OpenAI’s documentation says it indicates that registering a safety block succeeded, but it does not prove all execution stopped and does not reverse earlier actions. In an agent runtime, this means the receipt of an alert should trigger reconciliation rather than celebration: check outstanding tool jobs, cancel what can be safely canceled, identify completed external side effects, preserve artifacts, and require human approval before rollback, cleanup, credential changes, or user-facing final decisions.

The opening implementation contract

The rest of this tutorial builds from a simple contract: use Responses API coverage where it applies, but never outsource your safety state machine to monitoring alone. Match HTTP 403 errors by the documented code misalignment_policy_violation rather than by message text. Stop further actions immediately. Do not automatically retry a blocked request. Preserve request IDs, response lineage, tool-call records, streaming checkpoints, webhook receipts, safety-alert IDs, and application audit records. Then route the case for operator review with the neutral assumption that the alert is a concern requiring investigation, not proof of wrongdoing.

That contract is deliberately conservative because it matches the failure modes OpenAI calls out in its API documentation: monitoring can miss issues, legitimate work can be flagged, streaming errors can arrive after output, a stopped request does not undo prior actions, webhooks can duplicate or arrive after retries, and alert retrieval is not a full audit history. A production implementation that respects those limits will be slower than a blind retry loop, but it will give developers, administrators, security teams, and reviewers the evidence they need to make defensible decisions when an agent crosses into consequential territory.

Build the request path so a safety block stops the agent, not just the API call

Implement OpenAI Misalignment Monitoring for Agents: Responses API Coverage, 403 Handling, Safety Webhooks, and State Reconciliation — first editorial explainer visual

The request path is the place where misalignment monitoring has the most operational value because it is the last point before the model response can be converted into tool calls, file changes, data transfers, or business decisions. OpenAI’s misalignment monitoring can flag concern in consequential contexts such as sensitive-data access, sensitive-data transfer, and destructive changes, but OpenAI also states that a flag is not proof of wrongdoing and that monitoring can miss issues or flag legitimate work. Your integration therefore needs two properties at the same time: it must stop unsafe continuation when the API returns a block, and it must preserve enough evidence for a human reviewer to distinguish a real policy issue from a legitimate task that needs a different approval path.

The defensive pattern in this section treats every Responses API call as part of an application-controlled transaction. Before the request is sent, the application creates a local operation record with a stable request lineage, a privacy-preserving safety identifier, a list of authorized tool capabilities, and an empty evidence envelope. During the request, streaming output is buffered but not dispatched to external systems until gates are satisfied. After the request, the application either commits approved tool work, freezes the operation for review, or records a clean failure. This design is important because OpenAI’s documentation warns that a stopped request does not undo actions that already happened and that there is no general resume mechanism for a stopped conversation.

Assign request lineage before sending the model request

Create your own immutable operation ID before calling the API, then store the OpenAI request and response identifiers when they become available. The application ID is what lets your database, queue, tool runner, webhook receiver, and operator console agree on the same unit of work even when the OpenAI response is interrupted, a webhook arrives later, or a worker retries a local database write. The OpenAI request or response ID is evidence that connects your operation to the API interaction; it should not be the only primary key for internal incident handling because some failures occur before a full response object is available.

Field When to create it Why it matters during a safety block
application_operation_id Before the OpenAI request Provides a durable key for logs, queued tool calls, reviewer notes, and rollback records even if the API request fails early.
safety_identifier Before the OpenAI request Associates safety telemetry with a stable end-user identifier without sending a direct personal identifier when a privacy-preserving value is sufficient.
conversation_lineage Before the OpenAI request Records whether the request uses persisted reasoning, a WebSocket continuation, OpenAI compaction, a durable conversation, manual replay, or another state mode.
openai_response_id When returned by the API Supports reconciliation with later state, logs, and safety alerts; store it append-only rather than overwriting earlier IDs.
tool_dispatch_batch_id Before executing any tool batch Separates generated intent from executed action, allowing the system to prove which actions were still pending when the block occurred.

Safety identifiers should be stable per end user, but they should not expose an email address, account name, phone number, or other direct personal identifier if a privacy-preserving hash or equivalent internal pseudonymous value will satisfy your policy. OpenAI’s safety best-practices guidance says safety identifiers do not automatically carry across APIs or sessions, so the application must attach the identifier consistently on every relevant request. Treat the mapping from the safety identifier back to the real user as sensitive application data with access controls, retention rules, and audit logging.

Separate model generation from tool dispatch

A safe agent runtime should not execute a tool call merely because a streamed token or response item suggests an action. Put a dispatch gate between model output and tool execution, and require the gate to check the current operation status, the safety-block status, the tool’s authorization scope, and any required human approval. External transfers, destructive cleanup, credential changes, irreversible incident actions, and consequential decisions must remain human-approved actions rather than automatic side effects of a model response.

The dispatch gate should treat pending tool calls as proposals until the operation is explicitly marked eligible for execution. For low-risk internal reads, eligibility might require only a completed response, a matching authorization policy, and a clean operation status. For sensitive data access, sensitive data transfer, or destructive changes, eligibility should require a recorded approval with the approver identity, approval time, reviewed evidence, and exact action boundary. A safety block at any point should flip the operation into a frozen state that prevents new tool dispatch, cancels queued-but-unstarted tool jobs, and requires operator review before any remaining work can proceed.

function startAgentOperation(user, conversationState, requestedTask):
    operation = createOperationRecord({
        status: "OPENAI_REQUEST_PENDING",
        safety_identifier: privacyPreservingStableId(user),
        conversation_lineage: describeStateMode(conversationState),
        requested_task_summary: summarizeForAudit(requestedTask),
        tool_policy_snapshot: currentToolPolicyVersion(),
        evidence: []
    })

    responseStream = null

    try:
        responseStream = sendResponsesRequest({
            operation_id: operation.id,
            safety_identifier: operation.safety_identifier,
            conversation_state: conversationState,
            task: requestedTask,
            stream: true
        })

        for event in responseStream:
            appendEvidence(operation.id, "openai_stream_event_metadata", safeMetadata(event))

            if eventContainsFinalResponseId(event):
                recordOpenAIResponseId(operation.id, event.response_id)

            if eventContainsOutputText(event):
                bufferOutputForReview(operation.id, event.text_fragment)

            if eventContainsToolProposal(event):
                recordToolProposal(operation.id, normalizeToolProposal(event))
                keepToolProposalPending(operation.id)

            if eventContainsStreamError(event):
                handleOpenAIError(operation.id, event.error)
                return freezeResult(operation.id)

        markOperationStatus(operation.id, "MODEL_RESPONSE_COMPLETE")
        evaluatePendingToolProposals(operation.id)
        return operationResult(operation.id)

    catch error:
        handleOpenAIError(operation.id, error)
        return freezeResult(operation.id)

This pseudocode deliberately buffers output and records tool proposals before execution. It does not include credentials, service endpoints, or tool-specific instructions because the safety behavior should be independent of any particular deployment. In production, the functions that append evidence and change operation status should use durable writes with transaction boundaries; if the process crashes after receiving a block but before updating a queue, a reconciliation worker must still be able to find pending tool work and freeze it.

Handle streaming as a partial-output protocol

Streaming clients need stricter safeguards than non-streaming clients because an error can arrive after some output has already been delivered to your application. OpenAI’s misalignment monitoring guidance explicitly warns that streaming errors can occur after partial output. The defensive rule is simple: streamed output may be displayed in a non-consequential draft surface if your product policy allows it, but it must not be treated as permission to dispatch tools, transfer data, alter records, send messages, or update a trusted system until the response has completed and the operation remains unblocked.

If your user interface streams text to a human operator, label partial output as provisional and bind it to the operation record. If the stream later fails with a safety block, the interface should replace action controls with a review state and show that the visible text is evidence, not an approved recommendation. This distinction prevents a common failure mode where a user copies partial instructions or an automation layer consumes partial JSON before the application has processed the final API status.

For structured outputs, avoid incremental execution of partially streamed objects. Buffer the object, validate it after stream completion, and then run the same authorization and approval checks you would run for a non-streaming response. If your agent uses a queue, enqueue tool proposals only after the operation status is still allowed; if proposals were enqueued earlier for latency reasons, the queue worker must re-check the operation status immediately before execution and must refuse work from frozen or review-required operations.

Parse HTTP 403 by code, not by message text

When OpenAI blocks a request before streaming, the documented response is HTTP 403 with an error of type invalid_request_error and code misalignment_policy_violation. Your integration must match the code exactly and should not depend on error-message wording, punctuation, localization, or display text. The correct response is stop, no automatic retry, preserve evidence, and route the operation for operator review.

function handleOpenAIError(operationId, error):
    parsed = parseErrorObject(error)

    appendEvidence(operationId, "openai_error_metadata", {
        http_status: parsed.http_status,
        error_type: parsed.error_type,
        error_code: parsed.error_code,
        request_id: parsed.request_id,
        response_id: parsed.response_id,
        received_at: now()
    })

    if parsed.http_status == 403 and
       parsed.error_type == "invalid_request_error" and
       parsed.error_code == "misalignment_policy_violation":

        markOperationStatus(operationId, "SAFETY_BLOCKED_REVIEW_REQUIRED")
        setRetryPolicy(operationId, "NO_AUTOMATIC_RETRY")
        cancelQueuedToolWork(operationId)
        freezePendingToolProposals(operationId)
        createOperatorReviewCase(operationId, {
            reason: "OpenAI misalignment monitoring block",
            required_review: [
                "original user instruction",
                "conversation lineage",
                "partial streamed output, if any",
                "tool proposals",
                "already-started tool actions",
                "authorization and approval records"
            ]
        })
        return

    markOperationStatus(operationId, "OPENAI_REQUEST_FAILED")
    applyNonSafetyFailurePolicy(operationId, parsed)

The exact match on misalignment_policy_violation is a safety control, not a general error handler. Network failures, rate-related failures, schema errors, and other application errors may have their own retry policies, but a misalignment block should not be retried automatically with a rewritten prompt, a new conversation, a different state strategy, or a bypassing tool configuration. Retrying automatically can convert a review signal into an evasion loop, and it can destroy the clean evidence needed to decide whether the original task was legitimate.

Stop means stop new execution, then reconcile what already happened

A block stops further execution in your application only if your application enforces that stop across queues, workers, browser sessions, file systems, and downstream services. OpenAI’s documentation cautions that a stopped request does not undo earlier actions. Therefore, the first incident action after a block is not “rollback everything”; it is “freeze new execution and inventory completed, in-flight, and pending actions.” Rollback, credential rotation, deletion, user notification, and external reporting may be necessary later, but those steps are consequential and should require the appropriate human approval and organizational policy checks.

Action state at time of block Immediate application behavior Reviewer question
Generated but not dispatched Freeze the proposal, prevent dispatch, and preserve the normalized tool arguments for review. Was the proposed action authorized by the user, policy, and tool scope?
Queued but not started Cancel or hold the queue job using the operation status as the authority. Could the job still run from another worker, scheduled retry, or duplicate message?
Started but not completed Attempt safe pause if supported, record the current state, and escalate before destructive intervention. Would stopping the operation cause data loss, partial writes, or external side effects?
Completed before the block Record outputs, destination, timestamps, approvals, hashes where applicable, and downstream identifiers. Did the completed action exceed the original authorization or disclosure boundary?
Unknown because telemetry is missing Mark as unresolved, preserve available logs, and require manual investigation before resuming. Which control failed to produce audit evidence, and how should it be fixed before re-enabling the path?

Reconciliation should be deterministic. Build a worker that scans for operations in SAFETY_BLOCKED_REVIEW_REQUIRED, finds all tool proposals and dispatch batches tied to the operation ID, checks each downstream job’s last durable state, and writes a single action ledger. Do not rely on the model to summarize what it did, and do not ask for private reasoning. Use observable evidence: original messages, response IDs, output items, encrypted reasoning items where required for continuity preservation, tool-call records, queue events, approval records, file metadata, database transaction IDs, and destination records allowed by your privacy policy.

Preserve evidence without expanding access

Evidence preservation must be append-only and access-controlled. Store the original user instruction, system and developer policy version identifiers, conversation-state mode, response output items needed for state reconstruction, tool proposals, tool execution records, partial streamed content, HTTP status, exact error code, timestamps, and operator actions. If a request used stateless reasoning-model state, OpenAI’s guidance says applications should preserve every item in the response output, including encrypted reasoning items and assistant phase values; those items are opaque continuity records and should not be represented to reviewers as raw chain-of-thought.

Evidence collection should not become a privacy incident. Redact or compartmentalize sensitive content according to your data policy, and ensure that reviewers can access the original evidence needed to verify a model-produced summary or decision without granting them unnecessary access to unrelated user records. If a credential appears to have been exposed or misused, OpenAI’s safety best-practices guidance recommends prompt revocation and replacement; do not reproduce the credential in tickets, chat messages, dashboards, or training examples, and do not delay revocation merely to keep an investigation artifact intact.

function reconcileBlockedOperation(operationId):
    requireStatus(operationId, "SAFETY_BLOCKED_REVIEW_REQUIRED")

    ledger = {
        operation_id: operationId,
        generated_proposals: listToolProposals(operationId),
        queued_jobs: listQueueJobs(operationId),
        started_actions: listStartedToolActions(operationId),
        completed_actions: listCompletedToolActions(operationId),
        approvals: listApprovalRecords(operationId),
        openai_ids: listOpenAIIdentifiers(operationId),
        unresolved_gaps: []
    }

    for job in ledger.queued_jobs:
        if job.status in ["READY", "DELAYED", "RETRY_WAITING"]:
            holdQueueJob(job.id, "safety review required")

    for action in ledger.started_actions:
        if action.supports_safe_pause:
            requestSafePause(action.id)
        else:
            ledger.unresolved_gaps.append({
                action_id: action.id,
                issue: "manual review required before intervention"
            })

    writeAppendOnlyLedger(operationId, ledger)
    notifyReviewTeam(operationId)
    return ledger

The reconciliation worker should never perform destructive cleanup, credential changes, external notifications, or irreversible incident response on its own. It can hold local jobs, request safe pauses, and gather evidence; it should route consequential steps to an operator workflow with explicit approvals. This boundary keeps the safety system from creating a second incident while responding to the first one.

Design the operator review case as a decision record

An operator review case should state that the API returned a misalignment-monitoring concern and should avoid declaring that the user, agent, or model violated policy until the evidence has been reviewed. The reviewer needs a compact timeline, the exact misalignment_policy_violation match, the request and response identifiers you received, the safety identifier, the conversation-lineage mode, partial streamed output if present, tool proposals, action ledger, approvals, and any known gaps. If alert details later arrive through webhook retrieval, attach them as additional evidence rather than overwriting the original block record.

The review workflow should provide a small set of explicit outcomes: false positive or legitimate work with controls satisfied; legitimate work requiring additional approval; policy violation or unauthorized action requiring containment; insufficient evidence requiring investigation; or product-control failure requiring engineering remediation. Each outcome should map to allowed next steps. For example, legitimate work requiring additional approval may create a new operation after approval, while unauthorized external transfer may trigger containment and notification procedures under your incident policy. Do not resume the stopped conversation as if it had merely paused, because OpenAI’s API guidance says there is no general resume mechanism for a stopped conversation.

Recommended operating rule: a misalignment_policy_violation block freezes the operation, not the user account. The case may involve legitimate work, ambiguous instructions, unsafe tool scope, partial execution, or an actual policy issue. Preserve the evidence, stop new agent actions, reconcile completed work, and let authorized humans decide the next step.

Test the request path with failure modes, not happy paths

Integration tests should prove that your application does not dispatch tools after a block, does not retry blocked conversations automatically, and does not lose evidence when streaming fails after partial output. Use simulated API error objects and synthetic tool proposals rather than real sensitive data or abusive scenarios. The test should assert exact matching on misalignment_policy_violation, correct operation status, queued-job cancellation or hold behavior, evidence persistence, and reviewer-case creation.

  • Pre-stream 403 test: simulate HTTP 403 with invalid_request_error and misalignment_policy_violation, then verify no tool dispatch occurs and retry policy is set to no automatic retry.
  • Partial-stream error test: deliver several harmless output fragments, then a simulated stream error with the exact code, and verify the partial content is marked provisional evidence rather than approved output.
  • Queued-job race test: enqueue a synthetic tool proposal, flip the operation to safety-blocked, and verify the worker checks operation status immediately before execution.
  • Completed-action reconciliation test: mark one synthetic action as completed before the block and verify the ledger records it without attempting automatic destructive cleanup.
  • Conversation-state preservation test: verify that response output items required for continuity and reconciliation are stored append-only and are not selectively rewritten to make the lineage look cleaner.

The strongest signal that this implementation is working is not a dashboard with zero alerts. It is a repeatable record showing that when the API returns the exact block code, the agent stops dispatching new actions, reviewers receive the original evidence, earlier side effects are inventoried, and the system does not attempt to evade the block through retry, prompt rewriting, state omission, or alternate tooling.

Implement the safety webhook receiver and reconcile interrupted agent state

Implement OpenAI Misalignment Monitoring for Agents: Responses API Coverage, 403 Handling, Safety Webhooks, and State Reconciliation — second editorial workflow visual

Identifier and privacy boundary: The webhook event ID is not the alert ID. The alert reason can be null, including for Zero Data Retention (ZDR), and alert retrieval is not a complete audit history.

Identifier and privacy boundary: The webhook event ID is not the alert ID. The alert reason can be null, including for Zero Data Retention (ZDR), and alert retrieval is not a complete audit history.

Identifier and privacy boundary: The webhook event ID is not the alert ID. The alert reason can be null, including for Zero Data Retention (ZDR), and alert retrieval is not a complete audit history.

OpenAI’s misalignment monitoring guide treats safety.alert.created as an asynchronous signal that a safety alert exists, not as a complete incident record and not as an automatic stop mechanism for every runtime path. The receiver therefore has two jobs: first, accept only authentic events and make durable receipt decisions quickly; second, retrieve and reconcile the alert against your own request lineage, tool ledger, conversation state, and approval workflow before any operator decides what to do next.

The most important design rule is that webhook handling must not become the place where the application performs consequential actions. OpenAI’s webhook guidance says receivers should preserve the raw request body for signature verification, verify the inbound signature using the project signing secret, return a 2xx response quickly for valid events, and move non-trivial work to a background process. For safety alerts, that background process should create or update a review case, retrieve alert details with same-project credentials, and then reconcile application state; it should not revoke credentials, delete files, roll back transactions, notify external parties, or resume the agent without human approval.

Receiver contract for safety.alert.created

Configure the webhook at the OpenAI project level that owns the monitored agent traffic, and treat the project boundary as part of your security model. OpenAI’s safety alert guide states that the webhook contains an alert ID rather than full alert details, and that details must be retrieved later using credentials from the same project with the api.safety.alerts.read permission. If your platform has separate projects for development, staging, and production, do not use a central service key from a different project to retrieve the alert; instead, route the event to the project-specific worker or credential vault that can perform the read under the correct authorization boundary.

Receiver requirement Implementation decision Operational warning
Preserve raw body Capture the unmodified request bytes before JSON parsing, compression changes, logging filters, or framework body transformations. Signature verification can fail if the receiver verifies a reserialized JSON object instead of the original body.
Verify signature Use the project webhook signing secret stored server-side and reject unverified events before enqueueing work. Never process an event merely because it contains a plausible type or alert ID.
Acknowledge quickly After verification and durable receipt, return a 2xx response without waiting for alert retrieval, case enrichment, or operator notifications. OpenAI retries failed deliveries with exponential backoff for up to 72 hours, and redirects are treated as failures.
Deduplicate Use the webhook-id header as the idempotency key for event receipt and queue insertion. Webhook delivery is not exactly-once; duplicate events must not create duplicate incident actions.
Separate identifiers Store the webhook event ID, the webhook-id header, and the safety alert ID in distinct fields. The webhook event ID is not the safety alert ID; confusing them makes retrieval, audit, and deduplication unreliable.

A durable receipt should be small, append-only, and sufficient for replay. Store the received timestamp, verification result, webhook-id, webhook event ID if present in the verified payload, event type, safety alert ID, project identifier, raw body hash, signature metadata needed for audit, and the queue job ID. Store the raw body itself only according to your organization’s retention and privacy policy; if you keep it, access should be restricted because webhook payloads are security-relevant records.

Sample receiver flow with raw-body verification and asynchronous work

The following example is intentionally SDK-neutral pseudocode. It shows the control flow you need, not a claim about a specific framework method name, endpoint path, or OpenAI SDK surface. Replace verifyOpenAIWebhookSignature, parseVerifiedJson, insertReceiptIfNew, and enqueueSafetyAlertJob with your framework’s verified implementation and your database primitives.

// Proposed receiver workflow: safety.alert.created
// Defensive purpose: verify, acknowledge, enqueue, and deduplicate.
// Do not perform destructive cleanup, credential changes, external notices,
// rollbacks, or agent resumption in this synchronous request handler.

async function receiveOpenAISafetyWebhook(request, response) {
  const receivedAt = now();

  // 1. Preserve the exact raw bytes before JSON parsing.
  const rawBody = await request.readRawBodyBytes();
  const headers = request.headers;

  // 2. Verify the signature with the project webhook signing secret.
  // The secret must be stored server-side and rotated if exposed.
  const verification = verifyOpenAIWebhookSignature({
    rawBody,
    headers,
    signingSecret: getProjectWebhookSigningSecret(request)
  });

  if (!verification.ok) {
    auditWebhookRejection({
      receivedAt,
      reason: "signature_verification_failed",
      remoteMetadata: minimalRequestMetadata(request)
    });
    return response.status(400).send("invalid webhook");
  }

  // 3. Parse only after verification.
  const event = parseVerifiedJson(rawBody);

  // 4. Deduplicate by the Standard Webhooks webhook-id header.
  const webhookId = headers["webhook-id"];
  if (!webhookId) {
    auditWebhookRejection({
      receivedAt,
      reason: "missing_webhook_id",
      eventType: event.type
    });
    return response.status(400).send("missing webhook id");
  }

  // 5. Accept only the event type this receiver is designed to process.
  if (event.type !== "safety.alert.created") {
    recordIgnoredVerifiedEvent({
      receivedAt,
      webhookId,
      eventId: event.id || null,
      eventType: event.type
    });
    return response.status(204).send("");
  }

  // 6. Extract the safety alert ID from the verified payload.
  // Keep this separate from event.id and webhook-id.
  const safetyAlertId = extractSafetyAlertId(event);
  if (!safetyAlertId) {
    auditWebhookRejection({
      receivedAt,
      webhookId,
      eventId: event.id || null,
      reason: "missing_safety_alert_id"
    });
    return response.status(400).send("missing alert id");
  }

  // 7. Create a durable receipt once. Duplicate deliveries should not
  // enqueue duplicate jobs.
  const receipt = await insertReceiptIfNew({
    receivedAt,
    webhookId,
    eventId: event.id || null,
    eventType: event.type,
    safetyAlertId,
    projectKey: identifyProjectFromVerifiedContext(request, event),
    rawBodyHash: sha256(rawBody),
    verificationKeyVersion: verification.keyVersion || null,
    status: "received"
  });

  if (receipt.inserted) {
    await enqueueSafetyAlertJob({
      receiptId: receipt.id,
      webhookId,
      safetyAlertId
    });
  }

  // 8. Acknowledge quickly after verification and durable receipt.
  return response.status(204).send("");
}

This receiver returns a non-2xx response only when it cannot verify or cannot form a valid durable receipt. A verified duplicate receives a 2xx response because the delivery has already been accepted. A verified but unsupported event type can also receive a 2xx response after being recorded as ignored, because retrying it will not make the receiver support that event. If your policy requires stricter rejection for unexpected event types, document that choice and confirm it does not create noisy retries that obscure real safety alerts.

Background retrieval with same-project credentials

The worker should retrieve the alert details after the receiver has acknowledged the webhook. OpenAI’s safety alert guidance says the webhook contains an alert ID rather than full details, and the retrieval must use a same-project key with api.safety.alerts.read. This design prevents the webhook handler from becoming a large synchronous dependency chain and gives you a controlled place to handle temporary retrieval failures, permission errors, and alert records that are incomplete from the application’s perspective.

// Proposed worker workflow: retrieve, enrich, and open review case.
// This worker may classify and prepare evidence, but human approval remains
// mandatory for consequential decisions.

async function processSafetyAlertJob(job) {
  const receipt = await loadWebhookReceipt(job.receiptId);

  if (receipt.status === "completed") {
    return;
  }

  const projectCredentials = await loadSameProjectCredentials({
    projectKey: receipt.projectKey,
    requiredPermission: "api.safety.alerts.read"
  });

  let alert;
  try {
    alert = await retrieveSafetyAlert({
      credentials: projectCredentials,
      safetyAlertId: receipt.safetyAlertId
    });
  } catch (error) {
    await recordAlertRetrievalFailure({
      receiptId: receipt.id,
      safetyAlertId: receipt.safetyAlertId,
      errorClass: classifyNonSecretError(error),
      retryable: isRetryableRetrievalError(error)
    });
    throw error; // Let the queue retry according to your worker policy.
  }

  await upsertSafetyReviewCase({
    receiptId: receipt.id,
    webhookId: receipt.webhookId,
    eventId: receipt.eventId,
    safetyAlertId: receipt.safetyAlertId,
    alert,
    normalizedReason: normalizeNullableAlertReason(alert.reason),
    requestPaused: alert.request_paused === true,
    status: "needs_operator_review"
  });

  await reconcileAgentStateForAlert({
    safetyAlertId: receipt.safetyAlertId,
    alert,
    receipt
  });

  await markWebhookReceiptCompleted(receipt.id);
}

Do not assume the alert reason is always populated. OpenAI states that alert reasons can be null, including for Zero Data Retention requests. Your database schema, case UI, metrics, and routing rules must therefore treat a null reason as a first-class state such as reason_unavailable, not as a parsing failure, a low-severity finding, or evidence that the alert is invalid. A useful operator view displays the alert ID, associated request identifiers if available, the null reason state, and the application evidence needed to reconstruct what the agent did before and after the signal.

Also record request_paused carefully. OpenAI’s misalignment monitoring documentation says request_paused=true indicates that registering a safety block succeeded, but it does not prove all execution stopped or earlier actions were reversed. In a distributed agent runtime, the API request, streaming client, planner, tool queue, browser automation worker, code runner, notification service, and data connector can all have separate state. The alert can tell you that OpenAI registered a pause for the monitored request path; only your own execution ledger can prove whether downstream dispatch was halted, already-started actions completed, or compensating work is required.

Deduplication model for 72-hour retries and worker replay

OpenAI’s webhook guide says failed deliveries are retried with exponential backoff for up to 72 hours and that duplicate events can occur. Your deduplication window should therefore last longer than the delivery retry horizon, and your database uniqueness constraint should be on webhook-id, not on alert ID alone. A single alert should generally map to one review case, but the event delivery system and the safety alert domain are different layers; keeping both identifiers gives you forensic clarity when investigating retries, receiver outages, or operator-visible duplicates.

Identifier What it represents How to use it What not to do
webhook-id header The delivery event identifier used for webhook idempotency. Use it as the unique key for durable receipt and queue insertion. Do not treat a missing dedupe record as proof no alert exists; retrieval and alert state are separate.
Webhook event ID The event object identifier from the verified webhook payload, when present. Store it for audit correlation and support investigations. Do not pass it to alert retrieval as though it were the safety alert ID.
Safety alert ID The identifier for the safety alert that must be retrieved in the background. Use it to fetch alert details with same-project credentials and to upsert the safety review case. Do not use it as the sole webhook dedupe key, because duplicate deliveries can share the same alert.
Response/request IDs Your application and OpenAI request lineage identifiers from the model interaction. Use them to connect the alert to conversation state, tool calls, user session, and operator evidence. Do not infer missing lineage from the alert alone; preserve it before every model request.

A robust queue worker should be idempotent even after the receiver deduplicates. Workers can crash after retrieving an alert but before updating the case, or after creating a case but before marking the receipt complete. Use upserts keyed by safety alert ID for the review case, append-only event tables for state transitions, and compare-and-set status updates for worker progress. If retrieval repeatedly fails because of credentials, permissions, or project mismatch, route the case to an operational dead-letter queue that is visible to administrators; do not silently drop the alert because the initial webhook was valid.

Reconcile tool actions that may already have happened

Misalignment monitoring is not a transaction manager for your tools. OpenAI’s guide is explicit that a stopped request does not undo earlier actions and that webhook alerts do not enable automatic stopping. Reconciliation must start from your tool ledger and determine which actions were planned, approved, dispatched, started, completed, failed, or pending at the moment of the block or alert. If your runtime cannot answer those questions, improve the runtime ledger before adding more autonomous tool capabilities.

For each safety review case, build a reconciliation snapshot with these fields: user safety identifier or other privacy-preserving user reference, application session ID, OpenAI response IDs, previous response lineage if used, conversation object ID if used, tool call IDs, tool names, authorization class, approval record, dispatch timestamp, external destination or internal resource class, idempotency key, completion status, result hash or summary, and any reversible state marker. This snapshot should be generated from application logs and tool records, not from a model-authored summary alone.

Tool action state Reconciliation action Human approval requirement
Planned but not dispatched Cancel the pending dispatch and attach the plan to the review case. Approval is required before any later re-dispatch if the action is consequential.
Dispatched but not confirmed complete Query your own tool status record or internal job state using existing safe observability paths. Approval is required before external follow-up, destructive cancellation, or irreversible rollback.
Completed internal read Record what data class was accessed, by which authorization, and whether output was exposed to the user or another system. Approval is required before widening access, notifying external parties, or changing credentials.
Completed external transfer Freeze further transfers, preserve destination metadata, and escalate according to data-handling policy. Explicit human approval is mandatory for takedown requests, deletion attempts, notifications, or legal/compliance actions.
Completed destructive change Preserve before/after evidence and identify whether an approved rollback procedure exists. Approval is mandatory before rollback or cleanup because those actions can create additional impact.

When the alert concerns sensitive-data transfer, sensitive-data access, or destructive changes, the operator needs original evidence rather than a model’s polished explanation. OpenAI’s safety best practices say human reviewers should have access to the original evidence needed to verify a model-produced summary or decision. In practice, this means the review case should show the relevant user instruction, the model output items preserved by your state layer, tool arguments after policy filtering, approval prompts and decisions, tool results, and immutable log references. Do not require reviewers to inspect private raw reasoning; OpenAI’s reasoning guidance describes reasoning items as opaque and not exposing raw reasoning text.

Reconcile conversation state without rewriting history

Conversation reconciliation is the process of deciding what state may be used for future interaction after a block or alert. OpenAI’s conversation-state documentation describes several ways to maintain Responses API state: manually replaying the complete response output, using a durable Conversation object, or using previous_response_id. Its reasoning guidance further says that for stateless reasoning-model requests, applications should preserve every item in the response output, including encrypted reasoning items and assistant phase values. These items support continuity but do not reveal private reasoning.

The operational rule is append-only first, resume later if approved. Preserve the full lineage that led to the alert, including the response output items you received before a streaming error, the encrypted reasoning items if present, assistant phase values, tool-call records, and application policy decisions. Do not manually rewrite, selectively omit, or “clean up” prior output items to make a later continuation look safer unless your governance policy explicitly records that a new branch was created, why it was created, who approved it, and what evidence was excluded from the runtime context.

  1. Freeze the affected run. Mark the agent session as safety-review pending so schedulers, cron continuations, browser workers, tool workers, and user-triggered retry buttons cannot dispatch new actions automatically.
  2. Identify the state mode. Record whether the run used manual replay, a Conversation object, previous_response_id, persisted reasoning, WebSockets, or OpenAI compaction, because coverage and continuation behavior differ across modes.
  3. Preserve all received output items. Store partial streaming output and structured response output exactly as received under your retention policy, including opaque reasoning continuity items where applicable.
  4. Map response lineage to tool lineage. Connect each tool call and tool result to the model response, user instruction, approval gate, and downstream job that produced it.
  5. Create a review branch if needed. If future work is allowed, start from an operator-approved state branch that explicitly excludes unsafe pending actions and includes a human-authored case note explaining the boundary.
  6. Do not auto-resume a stopped conversation. OpenAI’s guide states that there is no general resume mechanism for a stopped conversation, so your product should present a review outcome, not a silent continuation.

If your application uses persisted reasoning, record the effective reasoning-context mode and the model-family compatibility assumptions that your system used at request time. OpenAI’s reasoning documentation says compatibility is model-family specific and that incompatible reasoning is omitted when switching families. This matters during reconciliation because a later operator-approved continuation may not have the same hidden continuity substrate as the blocked request. Your case record should distinguish “we preserved the opaque continuity items” from “a later request actually reused them,” because those are different facts.

Case statuses and operator decisions

A safety alert case should use statuses that reflect evidence handling rather than moral certainty. OpenAI’s misalignment monitoring guide says a flag indicates a need for review and does not prove a policy violation or conclusively establish misalignment. Use labels such as received, alert_retrieved, reconciliation_pending, needs_operator_review, approved_to_continue_with_constraints, closed_no_action, escalated_security, or escalated_compliance. Avoid labels like confirmed_bad_agent unless a human investigation has actually established that conclusion under your incident policy.

The operator decision record should contain the alert ID, webhook receipt, request lineage, action ledger, conversation-state mode, whether request_paused was true, whether alert reason was null, what evidence the reviewer examined, what actions were frozen, and what next step was approved. If a credential may have been exposed or misused, OpenAI’s safety best practices recommend prompt revocation and replacement; however, the decision to rotate production credentials can be consequential and should follow your emergency credential procedure with an accountable human approver. Do not display compromised secrets in the case record, and do not delay urgent revocation merely to preserve a more convenient investigation narrative.

Operational recommendation: Treat the webhook as the starting whistle, not the incident report. The verified event tells you to retrieve the alert and inspect your own state; your request lineage and tool ledger tell you what the agent actually did; your human review process decides whether to continue, constrain, roll back, revoke, notify, or escalate.

Test the receiver and reconciliation path before enabling autonomous tools

Test cases should simulate duplicate deliveries, delayed deliveries within the 72-hour retry horizon, missing or malformed webhook-id, signature failure, unsupported event type, alert retrieval permission failure, same alert delivered more than once, null alert reason, request_paused=true with already-completed tool actions, request_paused=false with a completed external action, and a streaming response that produced partial output before an error. Each test should assert that the receiver never processes unverified events, never performs consequential work synchronously, deduplicates by webhook-id, preserves separate event and alert identifiers, and creates exactly one operator-visible review case per alert policy.

Finally, test state reconciliation as a product behavior, not just as a backend job. The user interface should avoid implying that the user violated policy or that the model conclusively acted wrongly. It should say that the agent run has been paused for review when your application has frozen it, identify any actions still awaiting approval, and prevent automatic retries that would send the same blocked or concerning request again. For administrators, the review console should show enough original evidence to verify the issue while respecting privacy, Zero Data Retention constraints, and the fact that alert retrieval is not a complete audit history.

Final verification plan: prove the safety path before you trust the agent

The final implementation milestone is not “the webhook receiver returns 200” or “the SDK throws an exception.” It is a repeatable evidence trail showing that a blocked continuation stops new dispatch, an alert-only request becomes reviewable, partial streams do not leak into tools, delayed webhooks are reconciled, and operators can decide what to do without reconstructing state from memory. OpenAI’s misalignment monitoring guide frames a flag as a concern requiring review, not as conclusive proof of wrongdoing, so your test plan must validate containment, state reconciliation, and human decision support rather than treating the monitor as a verdict engine.

Run these tests in an isolated environment with harmless tool doubles, synthetic records, and no real external transfers. The goal is to simulate the control flow around OpenAI-documented events and errors without creating credential exposure, public uploads, destructive changes, or unauthorized access. If a test requires a consequential operation to be represented, use a stub that records the intended action and an idempotency key, then require a human reviewer to approve any equivalent production action later.

Acceptance criteria by scenario

Scenario Test condition Required system behavior Acceptance criteria
Blocked continuation A Responses API continuation returns HTTP 403 with invalid_request_error and code misalignment_policy_violation. The agent marks the conversation as safety-stopped, prevents further tool dispatch, preserves request and response lineage, and creates an operator review case. No automatic retry occurs; no queued tool call created after the block executes; the review case includes the error code, request identifier, conversation lineage, pending tool queue, and last committed application state.
Alert without stop A Responses API request that is monitored but not automatically blocked later produces a safety.alert.created webhook. The webhook receiver verifies the signature, acknowledges quickly, deduplicates by webhook-id, and schedules background alert retrieval with same-project credentials. The system does not claim the request was stopped; the case status states “alert received, execution state under reconciliation”; any consequential continuation is held for human review under your own policy.
Partial stream A streaming response emits partial assistant output and then terminates with an error. The runtime treats the streamed text as incomplete, blocks tool execution derived from unfinalized output, and records which chunks were displayed, buffered, discarded, or committed. No destructive or external action is dispatched from partial output; user-visible text is labeled as interrupted if retained; the state log distinguishes received stream fragments from a completed model response.
Duplicate webhook The same webhook delivery is received more than once. The receiver uses the webhook-id header as an idempotency key and stores one durable receipt before background processing. Only one case transition is created for the duplicate event; repeated deliveries are logged as duplicates without re-running consequential actions.
Delayed webhook A webhook arrives after the related request has completed and after subsequent application events have occurred. The reconciliation worker links the alert to stored request lineage and examines all tool actions, user messages, and assistant outputs that occurred after the monitored request. The case timeline shows the alert receipt time, original request time, subsequent actions, and the operator decision about whether to pause, notify, roll back, or continue.
Unavailable alert details Background retrieval succeeds but some alert fields, including reasons, are null or unavailable. The case remains reviewable using application-side records and does not fabricate an explanation for the alert. The reviewer interface displays “reason unavailable” or equivalent neutral wording; escalation relies on original prompts, outputs, tool logs, identifiers, and policy context rather than invented detail.
Incompatible reasoning-family change The application switches model families after using persisted reasoning or opaque reasoning items. The state reconciler records that compatible persisted reasoning may not carry forward and preserves the complete observable output lineage. No operator or user interface claims that opaque private reasoning was recovered; the next request is built from supported conversation state and append-only records, not from a manually edited reconstruction.
Orphaned tool result A tool finishes after the model request was blocked, interrupted, or superseded. The tool result is stored as orphaned or quarantined until a reviewer decides whether it belongs to the active conversation state. The result is not silently injected into the next model turn; the case records tool call ID, idempotency key, completion time, side-effect class, and reviewer disposition.
Irreversible prior action A safety block or delayed alert is associated with a prior action that cannot be undone, such as an external transfer, destructive cleanup, credential change, or consequential user decision. The incident is escalated to the correct human role, evidence is preserved, and the system prevents automated “cleanup” that could destroy audit value. No automated reversal is attempted; notification, remediation, credential rotation, legal review, or user outreach happens only after explicit human approval under your incident policy.

Logging fields that make review possible

Logging must be append-only for the safety path because a stopped request does not undo earlier actions and OpenAI states there is no general resume mechanism for a stopped conversation. Record enough data to reconstruct causality without storing unnecessary personal data or raw private reasoning. OpenAI’s safety best practices recommend privacy-preserving safety identifiers where appropriate, so use stable end-user correlation keys that avoid direct personal identifiers when your policy allows it.

{
  "case_id": "internal durable review case identifier",
  "project_id": "OpenAI project or internal project mapping",
  "safety_identifier": "privacy-preserving stable end-user identifier when available",
  "conversation_id": "application conversation or thread identifier",
  "response_id": "OpenAI response identifier when available",
  "previous_response_id": "lineage pointer when used",
  "request_started_at": "timestamp",
  "request_completed_at": "timestamp or null",
  "effective_state_mode": "manual replay, conversation object, previous_response_id, or other supported mode",
  "effective_reasoning_context_mode": "persisted, stateless with preserved items, omitted due to incompatibility, or not applicable",
  "model_family_change": "true, false, or unknown",
  "http_status": "403 or other status when applicable",
  "openai_error_type": "invalid_request_error when applicable",
  "openai_error_code": "misalignment_policy_violation when applicable",
  "stream_status": "not_streamed, completed, interrupted_after_partial_output",
  "partial_output_commit_status": "not_displayed, displayed_as_interrupted, discarded, or quarantined",
  "webhook_id": "delivery idempotency key from webhook header",
  "webhook_event_id": "event identifier",
  "safety_alert_id": "alert identifier from the event payload",
  "request_paused": "true, false, or unknown",
  "alert_reason_available": "true or false",
  "alert_reason_summary": "operator-visible neutral summary or null",
  "tool_call_ids": ["application tool call identifiers"],
  "tool_idempotency_keys": ["application idempotency keys"],
  "tool_side_effect_class": "none, read_only, internal_write, external_transfer, destructive, credential_change, or consequential",
  "orphaned_tool_results": ["tool results completed after interruption or block"],
  "operator_case_status": "new, triaged, contained, awaiting_approval, resolved, or escalated",
  "human_approvals": ["approval records for irreversible or external actions"],
  "evidence_manifest": ["prompt/output/tool/log artifacts preserved under access control"]
}

Do not log opaque reasoning items as if they were readable chain-of-thought. OpenAI’s reasoning and conversation-state guidance treats reasoning items as preserved state artifacts that can be opaque and family-specific. Your records should preserve the full response output items required for supported state continuity, while the reviewer interface should rely on observable messages, tool calls, outputs, IDs, timestamps, and application policy evidence.

Incident roles for safety alerts and stopped continuations

Assign incident roles before enabling autonomous or semi-autonomous tools because webhook delivery can be delayed, duplicate, or incomplete. The on-call engineer owns containment of the runtime queue, verification that no unapproved tool dispatch is continuing, and preservation of raw webhook bodies needed for signature verification. This role should not make legal, privacy, or user-notification decisions alone unless your organization has explicitly delegated that authority.

The safety reviewer owns the concern-versus-proof judgment. This reviewer examines the original user request, model outputs, tool-call plan, application policy, alert metadata, and any unavailable-detail caveats. The reviewer’s decision record should distinguish “monitor concern confirmed,” “legitimate task with sufficient authorization,” “insufficient evidence,” and “requires specialist escalation,” because treating every alert as proven misconduct can break legitimate workflows while treating every alert as harmless defeats the control.

The service owner owns product-state reconciliation. This person decides whether to pause the user workflow, reissue a safe request from an earlier state, ask the user for clarification, or retire the task. For incompatible reasoning-family changes, the service owner must confirm that the next request is reconstructed through supported conversation-state mechanisms rather than by selectively rewriting history to obtain the desired outcome.

The security or privacy lead owns sensitive-data, credential, unauthorized-transfer, and destructive-action implications. If records show exposed or misused credentials, OpenAI’s safety best-practices guidance recommends prompt revocation and replacement, but credential changes are still consequential operational actions and must follow your approval procedure. If records show potential external disclosure, do not rely on a model-generated explanation; preserve evidence and use established privacy, legal, and incident-response channels.

The human approver owns irreversible decisions. External transfers, destructive cleanup, credential rotation, account lockouts, production rollback, user notification, regulator contact, legal preservation, or other consequential outcomes should not be performed automatically by a webhook worker or model-generated remediation plan. The implementation should make approval easy to record and hard to bypass.

No-general-resume warning for operators and developers

Operational warning: Do not build a “resume anyway” button that blindly continues a stopped conversation. OpenAI documents that a stopped request does not reverse earlier actions and that the API offers no general resume mechanism for a stopped conversation. Treat every continuation after a block as a new, reviewed decision based on reconciled state, preserved evidence, and explicit human approval where consequences exist.

A safe post-block path starts by freezing new tool dispatch, listing completed and pending actions, classifying side effects, and deciding whether the conversation can continue from an earlier clean state. If the prior request used persisted reasoning or preserved opaque reasoning items, do not assume that switching to another reasoning family carries the same context forward. If the next request is necessary, construct it from supported state records and a concise operator-approved summary of observable facts, not from hidden reasoning or edited fragments.

Release checklist before production enablement

  • Coverage documented: Your runbook states which flows use Responses API continuations that can be blocked, which flows may only alert, and which legacy or separate flows are outside this monitoring system.
  • 403 handling verified: Tests prove that misalignment_policy_violation is matched by code, automatic retry is disabled, and downstream dispatch stops.
  • Streaming guarded: Partial output cannot trigger tools, external messages, destructive actions, or user-facing claims that depend on completion.
  • Webhook receiver hardened: Raw body verification, fast acknowledgement, server-side signing-secret storage, duplicate handling, asynchronous processing, and dead-letter review are in place.
  • Alert reconciliation implemented: The event ID, webhook delivery ID, safety-alert ID, request lineage, and application case ID are stored as distinct fields.
  • Unavailable details tolerated: Null alert reasons, including cases associated with Zero Data Retention, do not break triage or cause invented explanations.
  • State lineage preserved: Full response output items needed for supported state continuity are stored according to policy, and incompatible reasoning-family changes are visible to reviewers.
  • Human approval enforced: External transfers, destructive cleanup, credential changes, irreversible remediation, and consequential user decisions require recorded approval.
  • Evidence access limited: Reviewers receive the original evidence needed for verification, but logs avoid unnecessary direct personal identifiers and do not expose secrets or private reasoning.

Conclusion: monitoring is useful only when the surrounding system can stop, remember, and review

OpenAI’s API misalignment monitoring gives agent builders an important signal in consequential contexts, but the integration succeeds only if your application treats that signal as part of a broader safety architecture. A 403 block must stop new execution; an alert-only webhook must become a durable review case; partial streams must remain non-authoritative; delayed and duplicate delivery must be normal operating conditions; and state reconciliation must be deterministic enough for a human reviewer to understand what happened.

The practical pattern is consistent across the whole tutorial: preserve lineage before the request, separate generation from tool dispatch, match documented error codes, verify webhooks from the raw body, deduplicate delivery, retrieve alert details with same-project authorization, and never assume that a stop undoes prior side effects. Where the monitor raises a concern, do not overstate it as proof; where the monitor is silent, do not treat silence as approval. Human approval, least privilege, idempotency, privacy-preserving identifiers, append-only logs, and incident roles remain the controls that make the monitoring signal actionable.

Access 40,000+ AI Prompts for ChatGPT, Claude & Codex — Free!

Get instant access to our complete Notion Prompt Library — the largest curated collection of prompts for ChatGPT, Claude, OpenAI Codex, and other leading AI models. Optimized for real-world workflows across coding, research, content creation, and business.

Access Free Prompt Library →

Useful Links

Get Free Access to 40,000+ AI Prompts for ChatGPT, Claude & Codex

Subscribe for instant access to the largest curated Notion Prompt Library for AI workflows.

More on this