25 ChatGPT-5.5 Prompts for Measuring Coding-Agent Research Productivity: Experiments, Task Success, Human Interventions, and Cost

Why coding-agent productivity needs a measurement system, not a vibes dashboard

OpenAI’s September 2026 research-acceleration article gives research leaders a useful but dangerous starting point for measuring coding-agent productivity. OpenAI says it has reached what it describes as an “automated research intern”: a supervised system that can complete well-defined research tasks that would take a skilled researcher a few days. The same article reports internal operating measures including 3.1 agent-workdays per human workday across its research organization, median researcher inference usage above $600 per day at API prices by mid-August 2026, 90th-percentile usage above $7,000 per day, and an all-time high in experiments per active experimenter since tracking began in January 2025. Those numbers are important because they show how intensively one frontier lab is instrumenting agent use; they are not independently audited productivity estimates and should not be copied into another organization as benchmarks without normalization.

This prompts article is designed for teams that want ChatGPT-5.5 to help build a measurement program around coding agents without making the common category error: treating activity as impact. A coding agent can spend tokens, run for hours, edit thousands of lines, launch experiments, and still fail to advance a research question. Conversely, a short agent session that deletes a flawed path, reproduces a negative result, or surfaces a safety blocker can create high value even when it produces little code. The prompts that follow should be used as measurement design tools, not as proof that a model, team, or workflow is productive by default.

The safest starting assumption is that agent work is supervised technical work, not autonomous scientific discovery. OpenAI’s own account preserves that distinction: people continue to set priorities, judge ideas and results, and decide whether to scale, pause, or deploy systems. OpenAI also states that high-level planning remains a minimal fraction of agent output tokens and that agents still require substantial human steering; more than half of successful four-to-eight-hour tasks involved at least one intervention. Any credible productivity analysis should therefore measure human intervention as part of the system, not as an embarrassing exception to be hidden from the dashboard.

For teams already building analytics workflows, make the measurement outputs reproducible as notebooks, spreadsheet schemas, SQL-ready tables, or review-ready summaries instead of leaving them as one-off chat transcripts. The goal is to make ChatGPT-5.5 reason over a complete evidence packet, identify missing fields, propose definitions, and generate decision rules that analysts can challenge before leaders act on the numbers. For deeper context on ChatGPT-5.5 Data Analysis Prompts, 30 ChatGPT-5.5 Prompts for Data Analysts: SQL Generation, Data Visualization, Statistical Analysis, and Automated Reporting is a practical companion. This article provides production-ready ChatGPT-5.5 prompts for data analysts covering SQL generation, data visualization, statistical analysis, automated reporting, and messy enterprise data workflows.

The eight measures leaders usually confuse

Coding-agent activity measures how often agents are invoked, how many sessions are running, which repositories or projects receive attention, and how many task requests enter the queue. Activity is useful for adoption and capacity planning, but it does not prove that useful research happened. A sudden rise in agent starts can mean the team found a productive workflow, or it can mean prompts are underspecified, tasks are repeatedly restarted, or engineers are using agents to compensate for broken internal tooling.

Token spending measures model consumption, usually as input, output, reasoning, tool-call, or total token usage depending on the telemetry available. OpenAI’s reported API-price inference figures show that token spend can become operationally material at research scale, but spending is still a cost signal rather than an outcome signal. A high-spend task can be justified if it eliminates a wrong research direction or produces a validated result; it is wasteful if it repeatedly regenerates boilerplate, loops through failing tests, or pursues low-priority experiments.

Runtime measures elapsed agent work time, wall-clock time, or normalized “agent-workdays,” depending on the organization’s definition. OpenAI’s 3.1 agent-workdays per human workday is an internal utilization metric, not a conversion rate between machine work and human labor. Runtime can reveal queue saturation, long-running investigations, or concurrency risk, but it cannot answer whether the work was correct, novel, reproducible, safe, or worth the compute.

Code volume measures files changed, lines added, lines deleted, commits proposed, pull requests opened, or artifacts generated. Code volume is easy to collect and easy to misuse. More code can mean useful implementation progress, but it can also mean churn, duplicated logic, overfitting to a local test, or unnecessary complexity. For research teams, code volume should be interpreted alongside review findings, reproducibility checks, experiment metadata, and whether the change clarified or obscured the underlying scientific claim.

Experiment count measures how many experiment runs, ablations, sweeps, benchmark evaluations, simulations, or validation jobs were completed. OpenAI reports that experiments per active experimenter reached an all-time high in August 2026 since tracking began, which suggests that agent-supported workflows can increase experimental throughput inside OpenAI’s environment. The operational warning is that experiment count is only valuable when paired with hypothesis quality, data validity, configuration capture, failure classification, and a rule for separating exploratory runs from decision-grade evidence.

Success rate measures the share of tasks completed against predefined acceptance criteria. This is stronger than raw activity, but it can still be gamed by shrinking task scope, choosing easy tickets, or marking incomplete work as “useful enough.” Because OpenAI says more than half of successful four-to-eight-hour tasks involved at least one human intervention, success-rate reporting should separate autonomous completion, assisted completion, rescued completion, and human-taken-over work. Otherwise, a dashboard can accidentally attribute human judgment to the agent.

Scientific progress measures whether the work changed what the team credibly knows. Examples include eliminating a hypothesis, improving a model under a valid evaluation, discovering a reproducible failure mode, confirming a safety concern, or producing an artifact that another researcher can independently inspect. Scientific progress is slower to score than activity, but it is the measure that matters most for research organizations. It requires review by domain experts, not just logs from the agent runtime.

Business value measures whether the research work supports an organizational goal such as faster product learning, lower engineering cost, safer deployment, better customer outcomes, defensible IP, reduced risk, or improved decision quality. Business value can lag the research activity by weeks or months, so it should not be forced into every task record. Instead, teams should tag likely value pathways and revisit them during portfolio review, especially when high inference spend or GPU allocation is involved.

The evidence packet to give ChatGPT-5.5 before asking for analysis

Before using the prompts in this masterclass, assemble an evidence packet that lets ChatGPT-5.5 distinguish effort, assistance, outcome, and value. A good packet does not need to expose secrets or personal data; it needs structured fields, consistent definitions, and enough context to prevent the model from inventing causal explanations. If sensitive repositories, credentials, customer data, or regulated records are involved, redact or aggregate before analysis and keep the source-of-truth logs inside approved systems.

Use when you need to turn these evidence categories into executive metrics, operating thresholds, and review cadences. The prompts in this article are deliberately stricter than a normal dashboard request: each one asks for inputs, method, output schema, assumptions, and a decision rule so that the model cannot simply summarize logs and call the result productivity. For deeper context on AI Agent KPI Framework, Enterprise AI in 2026: How Multi-Agent Systems Are Delivering Real ROI is a practical companion. This article explains how enterprises deploy multi-agent AI systems in 2026 to achieve measurable ROI, with implementation strategies, case studies, and cost-benefit analysis.

Recommendation: treat every coding-agent productivity report as a supervised-system report. The unit of analysis is not “the agent did work”; it is “a human-agent workflow attempted a defined task under a known configuration, with recorded interventions, reviewed outcomes, and attributable costs.”

The next sections provide 25 sequential prompts for building that measurement system. Run them in order if you are starting from raw logs, or selectively if you already have a taxonomy, cost model, intervention log, or review rubric. For each prompt, replace bracketed placeholders with your organization’s evidence packet, require the model to state uncertainty, and reject any output that converts OpenAI’s internal measurements into universal benchmarks for your team.

Prompts 1–9: Build the measurement spine before comparing agent output

These first nine prompts establish the measurement layer for coding-agent research productivity before any leader asks whether agents are “faster,” “cheaper,” or “better.” OpenAI’s September 2026 research-acceleration post reports internal measures such as more than $600 per day of median researcher coding-agent inference at API prices, 3.1 agent-workdays per human workday, and more than half of successful four-to-eight-hour tasks involving at least one human intervention. Those figures are useful as examples of what a mature measurement program can track, not as transferable benchmarks for another lab, company, or product team.

Use these prompts with a real evidence packet: task logs, experiment records, code-review outcomes, compute records, inference accounting, human-intervention notes, and reproducibility artifacts. If your source data is inconsistent, start with a data-quality audit before you ask ChatGPT-5.5 to calculate productivity; a weak schema will produce confident-looking but non-actionable ratios. For teams formalizing their evidence standard, treat missing task definitions, unlabeled interventions, and unlinked experiment outcomes as blockers rather than nuisances. For deeper context on Research Data Quality, How ChatGPT Search Actually Works in 2026: Understanding AI-Powered Web Results, Source Attribution, and When to Use It Over Google is a practical companion. This guide to ChatGPT Search focuses on source attribution, evidence provenance, and when AI-powered web results should be checked against conventional search, reinforcing the data-quality controls required in research metrics.

1. Prompt: Define non-transferable productivity metrics

Use case: Use this prompt when leadership wants one metric that summarizes coding-agent productivity. The goal is to separate operational measures, such as task count or agent runtime, from evidence of research progress, such as replicated experiment results or accepted code changes.

Copy-ready prompt:

You are helping design a coding-agent research productivity measurement system.

Inputs:
- Organization context: [TEAM / LAB / PRODUCT AREA]
- Research workflow: [HOW IDEAS BECOME EXPERIMENTS, CODE, RESULTS, DECISIONS]
- Available logs: [AGENT RUNTIME, INFERENCE COST, TASK RECORDS, HUMAN INTERVENTIONS, EXPERIMENTS, REVIEWS]
- Reporting audience: [EXECUTIVES / RESEARCH LEADS / FINANCE / SECURITY / PLATFORM]
- Measurement period: [DATES]

Method:
1. Define each proposed metric in plain language and as a formula.
2. Classify each metric as activity, throughput, quality, cost, human effort, or decision impact.
3. State what each metric cannot prove.
4. Identify which metrics are unsafe to compare across teams without normalization.
5. Create a decision rule for when a metric is reportable.

Output schema:
- Metric name
- Formula
- Required data fields
- Valid interpretation
- Invalid interpretation
- Normalization needed
- Minimum evidence threshold
- Decision rule

Assumptions:
- Do not treat agent runtime as human labor.
- Do not treat inference spend as productivity.
- Do not treat internal OpenAI measurements as transferable benchmarks.

Inputs: Provide raw field names from your logs, not only dashboard screenshots. Include whether costs are estimated at API prices, internal transfer prices, or vendor invoices.

Expected output: A metric dictionary that distinguishes activity from outcome and marks unsafe ratios before they enter executive slides.

Verification: Confirm that every metric has a denominator, exclusion rule, and interpretation warning. Reject any definition that equates agent-workdays with researcher-days.

2. Prompt: Build a task taxonomy for coding-agent research work

Use case: Use this prompt when task logs contain vague labels such as “debug,” “run eval,” or “fix experiment.” A taxonomy makes later success-rate and intervention analysis meaningful.

Copy-ready prompt:

Create a task taxonomy for coding-agent research productivity analysis.

Inputs:
- Sample task titles and descriptions: [PASTE 30-100 TASKS]
- Repository or experiment areas: [LIST]
- Typical task duration bands: [MINUTES / HOURS / DAYS]
- Human roles involved: [RESEARCHER, ENGINEER, REVIEWER, SECURITY, PLATFORM]
- Known task outcomes: [MERGED, REJECTED, RE-RUN, PAUSED, ESCALATED, UNKNOWN]

Method:
1. Group tasks into mutually exclusive primary categories.
2. Add secondary tags for risk, ambiguity, dependency, and reproducibility burden.
3. Mark which task types are suitable for coding agents and which require human ownership.
4. Identify task types where success cannot be judged from code completion alone.
5. Propose a labeling guide with examples and edge cases.

Output schema:
- Primary category
- Definition
- Inclusion criteria
- Exclusion criteria
- Secondary tags
- Agent suitability
- Human accountability point
- Example tasks
- Ambiguous cases

Decision rule:
A category is usable only if two independent labelers could apply it consistently from the available record.

Inputs: Include failed, paused, and abandoned tasks, not only completed work. A taxonomy trained on wins will understate uncertainty and human steering.

Expected output: A coding-agent task taxonomy that supports later comparisons by task type instead of mixing trivial refactors with multi-day research investigations.

Verification: Sample 20 historical tasks and label them using the taxonomy. If reviewers disagree on more than a small minority, revise category definitions before running productivity analysis.

3. Prompt: Construct a pre-agent baseline

Use case: Use this prompt before claiming productivity gains. A baseline should represent how comparable work moved through the organization before coding-agent adoption or before a specific workflow change.

Copy-ready prompt:

Design a defensible baseline for measuring coding-agent research productivity.

Inputs:
- Agent adoption date or rollout phases: [DATES]
- Historical task records before adoption: [SOURCE]
- Current task records after adoption: [SOURCE]
- Team composition changes: [HIRES, REORGS, ROLE CHANGES]
- Infrastructure changes: [COMPUTE, DATASETS, CI, EVALS, TOOLING]
- Research program changes: [MODEL CLASS, PROJECT PRIORITIES, SAFETY PAUSES]

Method:
1. Identify candidate baseline windows.
2. Exclude periods with major confounders unless they are explicitly modeled.
3. Match tasks by taxonomy, complexity, and outcome type.
4. Define baseline metrics for duration, human time, experiment throughput, review load, and rework.
5. List threats to validity.

Output schema:
- Baseline period
- Justification
- Included task types
- Excluded task types
- Matching variables
- Baseline metric values
- Known confounders
- Confidence rating
- Decision rule for using or rejecting the baseline

Assumptions:
Internal measurements from another organization can inspire metric design but must not be used as your baseline.

Inputs: Provide organizational events that could change productivity independently of agents, including compute allocation shifts, incident pauses, new evaluation harnesses, or repository migrations.

Expected output: A baseline plan that compares like with like and flags where causal claims are not supportable.

Verification: Ask a skeptical research lead to identify one alternative explanation for each apparent gain. If the baseline cannot address those explanations, report association rather than impact.

4. Prompt: Measure experiment throughput without confusing volume with progress

Use case: Use this prompt when experiment counts are rising and leaders want to know whether the increase reflects real research acceleration. OpenAI reports that experiments per active experimenter reached an all-time high in August 2026 since its tracking began in January 2025, but that is an internal activity measure, not an audited proof of scientific output.

Copy-ready prompt:

Analyze experiment throughput for a coding-agent-assisted research organization.

Inputs:
- Experiment records: [ID, OWNER, START, END, STATUS, HYPOTHESIS, RESULT]
- Active experimenter roster by period: [NAMES OR ANONYMIZED IDS]
- Agent involvement fields: [NONE / ASSISTED / AGENT-LED UNDER SUPERVISION]
- Human review records: [APPROVALS, INTERPRETATIONS, DECISIONS]
- Reproducibility artifacts: [SEEDS, CONFIGS, CODE, DATA VERSION, LOGS]
- Business or research decisions caused by experiments: [SCALE, PAUSE, DEPLOY, ABANDON]

Method:
1. Calculate experiments per active experimenter by period.
2. Split counts by taxonomy, agent involvement, and experiment status.
3. Separate launched, completed, interpretable, reproduced, and decision-relevant experiments.
4. Identify duplicate, low-quality, or non-comparable runs.
5. Propose a throughput dashboard with quality gates.

Output schema:
- Period
- Active experimenters
- Launched experiments
- Completed experiments
- Reproducible experiments
- Decision-relevant experiments
- Agent-assisted share
- Quality warnings
- Decision rule

Inputs: Include canceled and inconclusive experiments because they affect queue health and human review load.

Expected output: A throughput table that makes clear whether the organization is producing more launches, more valid results, or more decisions.

Verification: Randomly inspect experiments counted as “decision-relevant” and verify that a documented human decision followed from the result.

5. Prompt: Normalize agent-workdays without equating them to human labor

Use case: Use this prompt when platform teams track cumulative agent runtime. OpenAI describes total research-organization agent runtime reaching 3.1 agent-workdays per human workday, but the article also preserves human supervision and does not present runtime as equivalent to human labor.

Copy-ready prompt:

Normalize agent runtime into an operational agent-workday metric without treating it as human labor.

Inputs:
- Agent session logs: [START, END, MODEL, TASK ID, OWNER, STATUS]
- Definition of one human workday for reporting: [HOURS, POLICY, REGION]
- Concurrent session policy: [LIMITS OR OBSERVED PRACTICE]
- Human workday counts by period: [ACTIVE RESEARCHERS OR WORKERS]
- Exclusions: [IDLE TIME, FAILED STARTS, PAUSED TASKS, TEST RUNS]
- Cost fields if available: [API-PRICE ESTIMATE, INTERNAL COST, INVOICE]

Method:
1. Define an agent-workday formula from runtime.
2. Define exclusions and idle-time handling.
3. Calculate agent-workdays per human workday.
4. Segment by team, task type, model, and success label.
5. Add warnings against labor-equivalence and cost-equivalence claims.

Output schema:
- Formula
- Included runtime
- Excluded runtime
- Agent-workdays
- Human workdays
- Ratio
- Segments
- Interpretation
- Invalid claims
- Decision rule

Decision rule:
Report the ratio only beside task-success, intervention, and evidence-quality metrics.

Inputs: Provide raw timestamps and pause states rather than rounded daily totals. Runtime inflated by stuck sessions will distort operational capacity.

Expected output: A normalized runtime measure that supports capacity planning without implying that an agent-workday is a researcher-day.

Verification: Recompute one reporting period manually from session logs and confirm that idle, paused, and failed sessions were handled according to the written rule.

6. Prompt: Create task-success labels with reproducibility evidence

Use case: Use this prompt to replace binary “done/not done” labels with success definitions tied to review, reproducibility, and research value. This is essential for , because later model or workflow comparisons depend on consistent labels. For deeper context on AI Evaluation Dataset Design, Multi-Agent Orchestration Playbook (2026 Edition) is a practical companion. This production multi-agent orchestration playbook covers evaluation, reliability, cost, security, and staged rollout, giving teams a broader system context for constructing representative agent evaluation sets.

Copy-ready prompt:

Design task-success labels for coding-agent research tasks.

Inputs:
- Task taxonomy: [PASTE OR SUMMARIZE]
- Task records: [ID, DESCRIPTION, OWNER, AGENT INVOLVEMENT, STATUS]
- Review records: [CODE REVIEW, RESEARCH REVIEW, SECURITY REVIEW]
- Reproducibility evidence: [COMMANDS, CONFIGS, SEEDS, DATASETS, LOGS]
- Outcome records: [MERGED, USED IN EXPERIMENT, DECISION MADE, REVERTED]
- Known failure modes: [FLAKY TESTS, SPURIOUS RESULT, UNSAFE CHANGE, UNINTERPRETABLE OUTPUT]

Method:
1. Define success, partial success, failure, blocked, and unknown.
2. Specify evidence required for each label by task category.
3. Separate agent completion from human acceptance.
4. Add rules for tasks that required intervention.
5. Produce a labeler guide and adjudication workflow.

Output schema:
- Label
- Definition
- Required evidence
- Disqualifying evidence
- Human review requirement
- Intervention handling
- Examples
- Decision rule

Assumption:
A task can be completed by an agent and still fail if the result is not reproducible, accepted, or useful.

Inputs: Include examples of reverted patches and irreproducible experiment runs so the labeler guide captures negative evidence.

Expected output: A success-label schema that lets teams compare task outcomes without hiding quality failures inside completion counts.

Verification: Have two reviewers independently label the same sample and adjudicate disagreements. Do not publish success rates until disagreement categories are resolved.

7. Prompt: Estimate human time saved and human time added

Use case: Use this prompt when stakeholders ask for time savings. Coding agents can reduce implementation time while adding review, prompt-writing, debugging, security, or reproducibility work.

Copy-ready prompt:

Estimate human time saved and human time added by coding-agent use.

Inputs:
- Comparable baseline tasks: [TASK IDS AND DURATIONS]
- Agent-assisted tasks: [TASK IDS, AGENT RUNTIME, HUMAN TOUCHPOINTS]
- Human activity logs: [PROMPTING, REVIEW, DEBUGGING, APPROVAL, REWORK]
- Interviews or estimates from task owners: [STRUCTURED NOTES]
- Task success labels: [SUCCESS, PARTIAL, FAILURE, BLOCKED, UNKNOWN]
- Confidence levels: [HIGH / MEDIUM / LOW]

Method:
1. Estimate baseline human time for each task type.
2. Estimate actual human time spent with agent assistance.
3. Split time into saved implementation time and added supervision time.
4. Exclude failed or unknown tasks from net-savings claims unless separately reported.
5. Produce ranges, not false precision.

Output schema:
- Task category
- Baseline human time range
- Actual human time range
- Human time saved
- Human time added
- Net estimate
- Evidence source
- Confidence
- Decision rule

Decision rule:
Only claim time savings where baseline comparability, success evidence, and human-time evidence are all at least medium confidence.

Inputs: Use calendar time and active human time separately. A task that finishes overnight may improve cycle time without reducing researcher attention.

Expected output: A range-based estimate that shows whether agents saved human effort, shifted effort to review, or mainly increased parallelism.

Verification: Compare estimated human time against at least a small sample of diary logs, ticket comments, or repository activity to detect optimistic recall bias.

8. Prompt: Log interventions as steering events, not failures by default

Use case: Use this prompt to design an intervention log that captures human steering without automatically downgrading the task. OpenAI reports that more than half of successful four-to-eight-hour tasks involved at least one intervention, which means intervention frequency can measure supervision burden rather than simple failure.

Copy-ready prompt:

Design an intervention logging schema for supervised coding-agent research tasks.

Inputs:
- Task logs: [TASK ID, OWNER, START, END, STATUS]
- Agent transcripts or summaries: [AVAILABLE FIELDS]
- Human messages or actions: [PROMPTS, CORRECTIONS, APPROVALS, TOOL CHANGES]
- Tool and environment events: [TESTS, BUILDS, ERRORS, PERMISSIONS]
- Safety or policy events: [PAUSE, ESCALATION, DENIAL, REVIEW REQUEST]
- Success labels: [CURRENT LABELS]

Method:
1. Define intervention types: clarification, correction, scope change, approval, safety stop, resource change, interpretation, escalation.
2. Define severity and timing fields.
3. Separate necessary supervision from error recovery.
4. Link interventions to task outcome and human time.
5. Produce a logging template and summary metrics.

Output schema:
- Intervention ID
- Task ID
- Timestamp
- Actor
- Type
- Severity
- Trigger
- Human time estimate
- Outcome effect
- Evidence link
- Decision rule

Assumption:
An intervention may be compatible with task success, but repeated high-severity interventions indicate supervision burden or task-design problems.

Inputs: Include approvals and review requests, not only corrective prompts. Otherwise the log will undercount normal human governance.

Expected output: A structured intervention schema that supports analysis of when agents need help, why they need it, and whether task packets are too ambiguous.

Verification: Audit several successful tasks and confirm that interventions are visible in the log. If successful tasks show zero steering despite transcript evidence, the schema is incomplete.

9. Prompt: Grade evidence quality before executive reporting

Use case: Use this prompt before publishing a dashboard, board update, or research-operations memo. Evidence quality grading prevents weak logs from being converted into strong productivity claims.

Copy-ready prompt:

Grade the evidence quality behind coding-agent research productivity metrics.

Inputs:
- Proposed metrics: [PASTE METRIC DICTIONARY]
- Source systems: [TASK TRACKER, REPOSITORY, AGENT LOGS, EXPERIMENT DB, COST SYSTEM]
- Data completeness report: [MISSING FIELDS, COVERAGE, KNOWN GAPS]
- Labeling process: [WHO LABELED, GUIDE USED, AGREEMENT CHECKS]
- Baseline method: [SUMMARY]
- Confounders: [TEAM, COMPUTE, TOOLING, PRIORITY, INCIDENTS]
- Intended claims: [WHAT LEADERS WANT TO SAY]

Method:
1. Score each metric for completeness, consistency, traceability, label quality, baseline validity, and confounder control.
2. Classify each intended claim as supported, partially supported, or unsupported.
3. Rewrite unsupported claims into accurate language.
4. Identify minimum fixes needed before publication.
5. Produce an executive-safe summary.

Output schema:
- Metric
- Evidence grade
- Main weakness
- Supported claim
- Unsupported claim
- Required fix
- Reporting status
- Decision rule

Decision rule:
If a metric lacks traceable task-level evidence, it may be used for internal debugging but not executive productivity claims.

Inputs: Provide the exact wording of proposed claims. “Agents increased productivity” requires different evidence than “agent-assisted experiment launches increased in August.”

Expected output: A claim-by-claim evidence review that distinguishes operational observations from causal productivity conclusions.

Verification: Check that every approved claim can be traced to a metric definition, source table, labeling rule, and baseline note. Remove claims that depend on inference spend, runtime, or task volume alone.

Prompts 10–18: Cost, concurrency, confounders, and reproducible outcomes

Use these prompts after you have defined task types, baselines, success labels, intervention logs, and evidence quality. OpenAI’s September 2026 research-acceleration article reports internal adoption measures such as median researcher coding-agent inference above $600 per day at API prices, 90th-percentile usage above $7,000 per day, and 3.1 agent-workdays per human workday, but those figures are not audited productivity estimates and should not be copied into another organization as benchmarks. The prompts below force the model to separate accepted outcomes from attempted work, capacity from queueing, and correlation from causation.

Measurement rule for this section: treat inference spend, agent runtime, and concurrent sessions as operational inputs. Do not label them as productivity unless they are connected to accepted, reproducible research outcomes with documented human review.

10. Prompt: Calculate cost per accepted outcome without hiding rejected work

This prompt is for finance, research operations, and platform leaders who need a unit-cost metric that includes failed runs, inconclusive outputs, retries, and human review. The goal is not to prove that agents are cheaper than researchers; the goal is to compute the API-price inference cost attached to outcomes that your organization actually accepted.

Act as a research-operations analyst. Build a cost-per-accepted-outcome model for coding-agent research work.

Inputs I will provide:
- Time window:
- Project or team:
- Task taxonomy:
- Total agent runs:
- Accepted outcomes:
- Rejected outcomes:
- Inconclusive outcomes:
- Agent inference cost at API prices:
- Human review hours:
- Human intervention count:
- Retry count:
- Excluded work, if any:

Method:
1. Separate attempted tasks from accepted outcomes.
2. Allocate inference cost across accepted, rejected, inconclusive, and retried work.
3. Report cost per accepted outcome using at least two views: direct accepted-run cost and fully loaded attempted-work cost.
4. Do not convert inference spend into employee compensation or productivity.

Output schema:
- Metric name
- Formula
- Numerator
- Denominator
- Included costs
- Excluded costs
- Result
- Interpretation
- Warning

Assumptions:
List every allocation assumption and missing data risk.

Decision rule:
Recommend whether this metric is ready for executive reporting, operational debugging only, or not reliable yet.

11. Prompt: Normalize inference prices before comparing teams or months

Inference-price comparisons are easy to distort when model mix, context size, retry policy, region, routing, caching, or API-price schedules change. Use this prompt before comparing one lab, month, or cohort against another, and pair it with your internal token ledger or when you need a deeper finance view. For deeper context on AI Token Cost Analysis, What AI Coding Tools Really Cost in 2026: Complete Guide to Hidden Expenses, Token Budgets, and ROI Calculation for Engineering Teams is a practical companion. This guide breaks down the real 2026 costs of AI coding tools, including hidden expenses, token budgets, and ROI calculations for engineering teams.

Act as a cost-normalization reviewer. Normalize coding-agent inference cost so that two periods or teams can be compared without pretending their workloads were identical.

Inputs I will provide:
- Comparison A:
- Comparison B:
- Model versions or aliases used:
- Token counts, if available:
- API-price basis:
- Context-window characteristics:
- Routing or gateway changes:
- Retry policy:
- Tool-use pattern:
- Accepted outcomes by task type:

Method:
1. Identify all price, model, routing, and workload differences.
2. Produce raw spend, spend per attempted task, spend per accepted outcome, and price-normalized spend.
3. State which differences cannot be normalized from the available data.
4. Avoid saying one team is more productive unless outcome quality and task mix are comparable.

Output schema:
- Comparison dimension
- Raw value A
- Raw value B
- Normalized value A
- Normalized value B
- Adjustment applied
- Residual bias
- Confidence level

Assumptions:
List price and workload assumptions explicitly.

Decision rule:
Classify the comparison as valid, directionally useful, or misleading.

12. Prompt: Measure concurrency without calling every parallel session throughput

Concurrent agent sessions can increase work-in-progress while also increasing contention, review burden, and failure amplification. OpenAI reports internal agent runtime as 3.1 agent-workdays per human workday, but that measure should be treated as capacity usage rather than proof that three days of human-quality research were produced for each human day.

Act as a concurrency measurement designer. Build a dashboard that distinguishes parallel agent activity from accepted research throughput.

Inputs I will provide:
- Number of researchers:
- Concurrent sessions per researcher:
- Agent runtime:
- Human workdays:
- Accepted outcomes:
- Blocked tasks:
- Tasks awaiting human review:
- Queue length:
- Intervention events:
- Reproducibility pass rate:

Method:
1. Calculate active sessions, peak concurrency, average concurrency, and agent-workday ratio.
2. Compare concurrency with accepted outcomes and review backlog.
3. Flag points where added concurrency creates queueing, duplicated work, or lower evidence quality.
4. Do not equate agent-workdays with human workdays.

Output schema:
- Metric
- Formula
- Current value
- Healthy range, if known
- Evidence required
- Operational risk
- Recommended action

Assumptions:
State whether concurrency limits are policy limits, infrastructure limits, or observed behavior.

Decision rule:
Recommend increase, hold, reduce, or investigate concurrency.

13. Prompt: Quantify queue time and reviewer bottlenecks

A research group can appear highly automated while the critical path remains human review, priority-setting, environment access, or reproducibility checks. This prompt measures elapsed time between task creation, agent start, agent completion, human review, acceptance, and follow-up action.

Act as a queue-time analyst for supervised coding-agent research.

Inputs I will provide:
- Task ID list:
- Created timestamp:
- Agent-start timestamp:
- Agent-finish timestamp:
- First human-review timestamp:
- Acceptance or rejection timestamp:
- Reviewer role:
- Task priority:
- Task type:
- Outcome label:
- Rework required:

Method:
1. Calculate wait-to-start, agent-runtime, wait-for-review, review-duration, and total cycle time.
2. Segment by task type, priority, reviewer, and outcome.
3. Identify whether the dominant bottleneck is agent capacity, human review, environment setup, or decision latency.
4. Avoid claiming that shorter cycle time means better science unless evidence quality is stable.

Output schema:
- Segment
- Median total cycle time
- Median queue time
- Median agent runtime
- Median review wait
- Acceptance rate
- Bottleneck diagnosis
- Evidence caveat
- Proposed operational fix

Assumptions:
List missing timestamps and imputation rules.

Decision rule:
Recommend whether to add agent capacity, reviewer capacity, clearer task packets, or no change.

14. Prompt: Estimate retry cost and separate useful iteration from waste

Retries are not automatically bad: they can be legitimate exploration, recovery from infrastructure failure, or necessary refinement after human steering. They become a measurement problem when teams report only the final accepted result and omit the inference cost, elapsed time, and human interventions consumed by the unsuccessful attempts.

Act as a retry-cost auditor. Estimate how much coding-agent cost and delay comes from retries, and classify retries by cause.

Inputs I will provide:
- Task IDs:
- Initial run IDs:
- Retry run IDs:
- Retry reason:
- Inference cost per run:
- Runtime per run:
- Human intervention before retry:
- Final outcome:
- Failure logs or notes:
- Infrastructure incidents:

Method:
1. Group all runs by original task.
2. Classify retries as model-output issue, ambiguous task packet, environment failure, dependency failure, safety or policy stop, reviewer-requested refinement, or unknown.
3. Calculate retry cost as a share of total inference cost and per accepted outcome.
4. Identify preventable retry categories without blaming valid scientific iteration.

Output schema:
- Retry category
- Run count
- Cost
- Runtime
- Accepted outcomes affected
- Preventability
- Evidence
- Recommended fix

Assumptions:
State how ambiguous retry reasons were handled.

Decision rule:
Recommend whether to change prompting, task scoping, environment reliability, approval policy, or logging.

15. Prompt: Build a confounder analysis before presenting productivity trends

If experiments per active experimenter rise after coding-agent adoption, the change may still be confounded by model upgrades, researcher mix, project phase, easier task selection, larger compute allocation, tooling improvements, or a temporary push from leadership. OpenAI reports that experiments per active experimenter reached an all-time high in August since its January 2025 tracking began, but the article frames this as internal measurement rather than independently verified causal proof.

Act as a skeptical measurement scientist. Identify confounders before we report coding-agent productivity trends.

Inputs I will provide:
- Productivity trend:
- Adoption timeline:
- Model versions:
- Compute allocation changes:
- Team composition:
- Project phases:
- Tooling changes:
- Policy or safety pauses:
- Task selection changes:
- Outcome-quality metrics:

Method:
1. List plausible confounders and the direction of bias each could introduce.
2. Separate variables that can be measured from variables that require qualitative review.
3. Propose stratified analyses, matched comparisons, or interrupted-time-series checks where feasible.
4. Do not infer causality from correlation.

Output schema:
- Claimed trend
- Potential confounder
- Bias direction
- Data needed
- Test or control
- Remaining uncertainty
- Reporting language

Assumptions:
State which confounders cannot be ruled out.

Decision rule:
Classify the claim as causal, associational, descriptive only, or unsupported. Use causal only if the evidence design justifies it.

16. Prompt: Evaluate compute-growth effects without treating more compute as better research

Compute allocation can change research output, agent availability, and failure modes at the same time. OpenAI described a two-week pause in reinforcement-learning work after the Hugging Face incident, Astra-specific restrictions on August 7, a 59.2% fall in Astra-class GPU allocation the following week, and a 17.2% rise in other model-class allocation that offset about 85% of the Astra-class decline; those figures show why compute shifts must be modeled before interpreting productivity curves.

Act as a compute-allocation analyst. Evaluate whether changes in compute supply may explain observed changes in coding-agent research activity.

Inputs I will provide:
- Compute allocation by model class:
- Agent runtime:
- Experiment count:
- Accepted outcomes:
- Queue time:
- Safety pauses or restrictions:
- Model substitutions:
- Task types affected:
- Time window before change:
- Time window after change:

Method:
1. Plot or tabulate compute allocation, runtime, queue time, and accepted outcomes before and after the change.
2. Identify substitution effects between model classes.
3. Separate capacity effects from quality effects and policy effects.
4. Avoid saying compute caused productivity changes unless the evidence supports that design.

Output schema:
- Compute change
- Affected workload
- Operational effect
- Outcome effect
- Substitution evidence
- Confounders
- Confidence
- Recommended next measurement

Assumptions:
List allocation units and whether they are comparable across model classes.

Decision rule:
Classify the compute change as likely material, possibly material, not material, or unmeasurable with current data.

17. Prompt: Segment researcher-adoption cohorts instead of averaging everyone together

Averages hide adoption dynamics. Early adopters may pick easier tasks, power users may have custom workflows, and late adopters may face onboarding friction; OpenAI’s internal report gives distributional inference-use examples such as median and 90th-percentile daily API-price usage, which is more informative than a single organization-wide mean.

Act as an adoption-cohort analyst. Segment coding-agent usage and outcomes by researcher cohort.

Inputs I will provide:
- Researcher list or anonymized IDs:
- First agent-use date:
- Daily inference spend at API prices:
- Agent-runtime measures:
- Task taxonomy:
- Accepted outcomes:
- Intervention counts:
- Review backlog:
- Training or enablement events:
- Team or discipline:

Method:
1. Define cohorts such as non-user, new user, occasional user, regular user, and heavy user using transparent thresholds.
2. Report distributions, not only averages.
3. Compare cohorts within similar task types and project phases.
4. Avoid treating heavy usage as higher productivity unless accepted outcomes and quality evidence support it.

Output schema:
- Cohort
- Definition
- Researcher count
- Median usage
- 90th-percentile usage
- Accepted outcomes
- Intervention rate
- Quality evidence
- Interpretation
- Caveat

Assumptions:
Document threshold choices and privacy protections.

Decision rule:
Recommend enablement, tooling support, policy review, or no action for each cohort.

18. Prompt: Score experiment quality and reproducibility before counting wins

Accepted outcomes should require more than a plausible agent-written summary. OpenAI emphasizes that humans continue to set priorities, judge ideas and results, and decide whether to scale, pause, or deploy systems; your measurement system should therefore require evidence artifacts such as code diffs, logs, configuration snapshots, independent reruns, reviewer notes, and known limitations.

Act as a research-quality reviewer. Create a reproducibility and experiment-quality score for coding-agent-assisted research outcomes.

Inputs I will provide:
- Experiment objective:
- Hypothesis:
- Repository or artifact references:
- Environment details:
- Data sources:
- Evaluation method:
- Agent-generated changes:
- Human interventions:
- Results:
- Independent rerun status:
- Reviewer decision:
- Known limitations:

Method:
1. Score hypothesis clarity, artifact completeness, environment reproducibility, evaluation validity, intervention traceability, and reviewer confidence.
2. Require a separate label for reproduced, partially reproduced, not reproduced, and not yet checked.
3. Identify whether the result is ready for follow-up experiment, internal reporting, scaling decision, or rejection.
4. Do not count unreproduced outputs as accepted scientific wins.

Output schema:
- Quality dimension
- Score
- Evidence
- Missing artifact
- Risk
- Reviewer note
- Final reproducibility label
- Recommended next action

Assumptions:
State minimum evidence required for acceptance.

Decision rule:
Accept only if the reproducibility label and quality score meet the threshold I define; otherwise classify as needs rerun, needs review, or reject.

Prompts 19–25: Dashboards, thresholds, failures, pause gates, and executive control

This final section turns the measurement spine into an operating system: a dashboard specification, anomaly detection, intervention thresholds, failed-task review, safety pause gates, executive reporting, and a measurement-system audit. The prompts deliberately avoid treating OpenAI’s internal figures as transferable benchmarks. OpenAI reported that its research organization reached 3.1 agent-workdays per human workday and that more than half of successful four-to-eight-hour tasks involved at least one intervention, but those are internal measurements, not audited productivity constants for another lab or company.

19. Prompt: Specify a dashboard that separates activity, evidence, cost, and decisions

A useful coding-agent dashboard must not collapse everything into “agent productivity.” It should show which tasks were attempted, which produced accepted evidence, which required human steering, which consumed inference budget, and which changed a research decision.

You are ChatGPT-5.5 acting as a research-operations measurement architect.

Inputs:
1. Team scope: [teams, repositories, projects, model access paths]
2. Time window: [daily, weekly, monthly]
3. Task table fields: [task_id, owner, task_type, start_time, end_time, agent_runtime, status]
4. Experiment table fields: [experiment_id, hypothesis, evidence_link, result, reproduction_status]
5. Intervention log fields: [task_id, timestamp, intervention_type, reason, minutes_spent]
6. Cost fields: [model, inference_cost_basis, request_count, tokens_or_runtime, team_key]
7. Decision fields: [scaled, paused, rejected, deployed, follow_up_required]

Method:
Design a dashboard that separates activity metrics from outcome metrics. Include tiles for attempted tasks, accepted outcomes, rejected outcomes, human interventions, reproducibility status, cost per accepted outcome, retry cost, reviewer queue time, and decision impact. Add filters for team, task type, model, repository, researcher cohort, and month.

Output schema:
Return a table with columns: Dashboard Panel, Primary Question, Required Fields, Metric Formula, Minimum Evidence, Owner, Refresh Cadence, Decision Trigger.

Assumptions:
List assumptions about missing logs, delayed reviews, shared tasks, pricing basis, and incomplete reproduction evidence. Do not treat agent runtime as human labor or inference spend as productivity.

Decision rule:
Flag any dashboard panel as "not decision-grade" if it lacks task identifiers, accepted/rejected status, human review evidence, or a stable cost basis.

Implementation guidance: build the dashboard around joinable identifiers rather than screenshots or anecdotal updates. If the organization already has an observability environment, this dashboard should sit beside engineering reliability views rather than replacing them; the measurement design can be mapped into an so platform teams can combine model routing, cost, task status, and review queues without losing the research context. For deeper context on AI Operations Dashboard, How to Embed Codex in an Internal Operations Dashboard with App-Server, MCP Tools, Streaming, and Human Approvals is a practical companion. This tutorial shows how to embed Codex into an internal operations dashboard using an app server, MCP tools, streaming, and human approval flows for safe downstream actions.

20. Prompt: Detect anomalies before they become productivity claims

Anomaly detection should catch sudden jumps in cost, retries, approvals, runtime, failure rates, or missing evidence before a team describes the month as a productivity breakthrough. OpenAI’s reported increase in experiments per active experimenter is useful context, but each organization needs its own baseline and confounder checks.

You are ChatGPT-5.5 acting as an anomaly-detection analyst for coding-agent research metrics.

Inputs:
1. Historical baseline period: [dates]
2. Current period: [dates]
3. Metrics: [attempted_tasks, accepted_outcomes, failed_tasks, interventions, retry_count, inference_cost, reviewer_queue_time, reproduction_pass_rate]
4. Segments: [team, task_type, model, repository, researcher_adoption_cohort]
5. Known events: [model changes, policy changes, incidents, freezes, staffing changes, deadline pushes]

Method:
Compare current metrics against the baseline by segment. Use robust comparisons such as median shift, percentile shift, week-over-week change, and missingness rate. Separate beneficial anomalies from risk anomalies. Treat any metric spike as unproven until checked against accepted outcomes and evidence quality.

Output schema:
Return a table with columns: Anomaly, Segment, Direction, Magnitude, Possible Explanation, Evidence Needed, Risk Level, Recommended Action.

Assumptions:
State whether seasonality, staffing, repository changes, deadline pressure, or cost-basis changes may explain the anomaly.

Decision rule:
Escalate anomalies when cost rises faster than accepted outcomes, intervention rates collapse unexpectedly, missing evidence rises, reproduction pass rate falls, or reviewer queue time masks apparent throughput.

Operational warning: a falling intervention rate can mean agents are improving, but it can also mean humans stopped logging interventions or reviewers are overloaded. Treat low-touch success as valid only when accepted outcomes, reproduction evidence, and reviewer notes remain complete.

21. Prompt: Set intervention thresholds without punishing useful human steering

OpenAI stated that substantial human steering remains part of successful agent work, including more than half of successful four-to-eight-hour tasks involving at least one intervention. That makes intervention thresholds a quality-control mechanism, not a simplistic failure counter.

You are ChatGPT-5.5 acting as a research manager designing intervention thresholds.

Inputs:
1. Task categories: [implementation, experiment setup, debugging, literature synthesis, evaluation, refactor]
2. Expected task duration bands: [under_1h, 1_4h, 4_8h, over_8h]
3. Intervention types: [clarification, scope correction, tool approval, failed test guidance, safety escalation, result interpretation]
4. Historical intervention rates: [by task type and duration band]
5. Outcome labels: [accepted, accepted_with_revision, rejected, inconclusive]
6. Evidence requirements: [tests, notebooks, logs, reviewer signoff, reproduced run]

Method:
Create threshold bands that distinguish normal steering from excessive rescue. Define green, yellow, red, and pause thresholds for number of interventions, cumulative human minutes, repeated same-cause interventions, and safety-related interventions.

Output schema:
Return a table with columns: Task Segment, Normal Steering Range, Yellow Threshold, Red Threshold, Pause Threshold, Evidence Required, Manager Action.

Assumptions:
State that interventions can increase quality and are not automatically failures. Identify segments where high intervention is expected because tasks are ambiguous or high impact.

Decision rule:
Do not penalize a task for a single high-value intervention that prevents invalid work. Escalate repeated interventions caused by unclear task packets, tool misuse, missing permissions, non-reproducible results, or safety uncertainty.

Recommended practice: review intervention thresholds monthly for each task category. A debugging task and an experiment-design task should not share the same intervention norm, because the first may require concrete test feedback while the second may require human judgment about hypotheses and scaling decisions.

22. Prompt: Run a failed-task review that finds system causes, not scapegoats

Failed tasks are the fastest way to improve the measurement system because they expose missing task definitions, weak evidence standards, broken tooling, ambiguous ownership, and false success criteria. The review should distinguish agent limitations from human briefing errors, infrastructure failures, and review bottlenecks.

You are ChatGPT-5.5 acting as a neutral failed-task review facilitator.

Inputs:
1. Failed task records: [task_id, owner, prompt, task_type, model, tools, duration, cost, status]
2. Evidence artifacts: [logs, diffs, tests, experiment outputs, reviewer comments]
3. Intervention history: [timestamps, reasons, human minutes, actions taken]
4. Failure label options: [incorrect_result, non_reproducible, scope_drift, tool_failure, permission_issue, unsafe_action_requested, insufficient_review, unclear_task_packet]
5. Business or research impact: [blocked experiment, wasted compute, delayed review, no impact]

Method:
Classify each failed task by proximate failure and likely system cause. Identify whether the task should be retried, decomposed, converted to a human-only task, postponed, or removed from the agent queue. Separate preventable failures from acceptable exploration.

Output schema:
Return a table with columns: Task ID, Failure Label, System Cause, Evidence, Preventability, Retry Decision, Required Fix, Owner, Due Date.

Assumptions:
List uncertainties caused by missing logs, unavailable artifacts, ambiguous prompts, or undocumented human decisions.

Decision rule:
Approve a retry only when the failure cause has a concrete correction, such as a smaller task packet, clearer success criterion, added test, changed tool permission, reviewer assignment, or explicit safety boundary.

Operational warning: do not hide failed tasks from cost-normalized metrics. If only accepted outcomes are counted, leaders will underestimate retry cost, reviewer load, and the amount of human steering needed to convert agent activity into usable research output.

23. Prompt: Define safety pause gates for supervised coding-agent research

OpenAI’s research-acceleration article describes humans continuing to decide whether to scale, pause, or deploy systems, and it also describes a two-week pause in reinforcement-learning work after the Hugging Face incident. That example supports a practical rule: pause gates belong inside productivity measurement because unsafe or uncertain acceleration is not operational success.

You are ChatGPT-5.5 acting as a safety-and-operations policy drafter for supervised coding-agent research.

Inputs:
1. Task environments: [local sandbox, cloud workspace, repository, experiment cluster, external service]
2. Allowed tools and prohibited tools: [list]
3. High-impact actions: [deployment, data deletion, credential use, external network action, large compute allocation]
4. Monitoring signals: [tool calls, approvals, diffs, tests, cost spikes, anomaly flags, reviewer comments]
5. Incident categories: [scope drift, unauthorized access attempt, non-reproducible result, unsafe instruction, anomalous tool use, monitoring uncertainty]
6. Resume authorities: [research lead, security lead, platform owner, executive sponsor]

Method:
Create pause gates that stop or slow work when predefined conditions occur. Include immediate pauses, manager-review pauses, security-review pauses, and executive-review pauses. Define what evidence is required to resume.

Output schema:
Return a table with columns: Pause Gate, Trigger, Immediate Action, Evidence to Preserve, Reviewer, Resume Criteria, Communication Required.

Assumptions:
State that pause gates may slow legitimate work and should be measured as part of operating cost, not hidden as overhead.

Decision rule:
Pause work when the agent requests or attempts a high-impact action without explicit authorization, when monitoring evidence is missing, when results cannot be reproduced, when tool use deviates from task scope, or when a reviewer cannot determine whether the task stayed within approved boundaries.

Recommended practice: record both the pause and the avoided risk. A pause that prevents an unauthorized external action, a non-reproducible result from entering a paper, or an unsafe deployment from advancing is a successful control, even if it lowers short-term throughput.

24. Prompt: Produce an executive report that preserves uncertainty

Executive reporting should communicate whether coding agents are changing research capacity, but it must preserve uncertainty about causality. OpenAI’s figures on internal inference use and agent-workdays show one organization’s adoption pattern; they should not be converted into employee-equivalent productivity or used as a compensation comparison.

You are ChatGPT-5.5 acting as an executive reporting analyst.

Inputs:
1. Reporting period: [dates]
2. Baseline period: [dates]
3. Core metrics: [accepted_outcomes, attempted_tasks, experiments_per_active_experimenter, agent_runtime, inference_cost, interventions, reproduction_pass_rate]
4. Quality metrics: [evidence_score, reviewer_acceptance, failure_labels, downstream_decisions]
5. Safety metrics: [pause_count, escalation_count, unsafe_or_out_of_scope_attempts, unresolved_anomalies]
6. Confounders: [staffing, deadlines, model changes, infrastructure changes, policy changes]

Method:
Write an executive summary that separates observed facts, plausible interpretations, unresolved questions, and recommended decisions. Include a confidence rating for every productivity claim. Avoid causal language unless the evidence design supports it.

Output schema:
Return:
A. Five-bullet executive summary
B. Metric table with current period, baseline, change, confidence, and caveat
C. Decision table with recommendation, evidence, risk, owner, and next review date
D. "Claims we are not making" section

Assumptions:
State the limits of the data, including missing logs, non-random task selection, model changes, and human-review bottlenecks.

Decision rule:
Reject any executive claim that equates inference spend with productivity, agent runtime with human labor, task volume with progress, or correlation with causation.

Implementation guidance: include a “claims we are not making” section in every executive packet. Examples include: “we are not claiming autonomous discovery,” “we are not claiming that agent-workdays equal human workdays,” and “we are not claiming that higher cost caused higher-quality research.”

25. Prompt: Audit the measurement system before trusting the trend line

The final prompt audits whether the measurement system itself is reliable. This matters because productivity dashboards can be gamed by shrinking task scope, avoiding hard problems, under-logging interventions, delaying failure labels, or moving review work outside the recorded workflow.

You are ChatGPT-5.5 acting as an independent auditor of a coding-agent research productivity measurement system.

Inputs:
1. Metric definitions: [all definitions and formulas]
2. Data sources: [task tracker, agent logs, cost logs, experiment registry, review system, incident log]
3. Sampling method: [full population or sample]
4. Governance rules: [who can edit labels, who approves outcomes, who resolves disputes]
5. Recent reports: [dashboard exports, executive summaries, anomaly reviews]
6. Known incentives: [team goals, budget pressure, publication deadlines, performance reviews]

Method:
Audit definition stability, data completeness, label integrity, evidence traceability, cost-basis consistency, confounder handling, and anti-gaming controls. Sample accepted, rejected, paused, and retried tasks. Compare dashboard values against raw artifacts.

Output schema:
Return a table with columns: Audit Area, Test Performed, Finding, Severity, Evidence, Required Remediation, Owner, Deadline. Then provide an overall rating: Reliable, Reliable with Exceptions, Not Decision-Grade, or Unusable.

Assumptions:
Identify data that could not be verified and explain how that affects confidence in reported productivity.

Decision rule:
Rate the system "Not Decision-Grade" if accepted outcomes cannot be traced to evidence, if intervention logs are optional or incomplete, if failed tasks are excluded from cost metrics, if metric definitions changed without annotation, or if reviewers can approve their own disputed outputs without oversight.

Recommended practice: run this audit before quarterly planning, not after a funding decision has already been made. The audit should test whether leaders can reproduce a reported trend from raw task records, cost logs, intervention notes, and experiment evidence without relying on a manually curated slide.

Implementation scorecard for coding-agent research productivity

Decision cadence for research leaders

Use a daily operational review for blocked tasks, safety pauses, missing approvals, and anomalous cost spikes. This review should be short and evidence-based: each item needs a task ID, owner, current blocker, next action, and escalation path.

Use a weekly research-operations review for accepted outcomes, failed-task causes, retry cost, intervention patterns, and reviewer bottlenecks. Weekly review is the right cadence for deciding whether to rewrite task packets, change routing policy, adjust reviewer staffing, or pause a problematic task class.

Use a monthly leadership review for trend claims, budget decisions, adoption cohorts, model or tooling changes, and evidence quality. Monthly reporting should compare against a stable baseline and annotate confounders such as staffing changes, deadlines, infrastructure migrations, and policy restrictions.

Use a quarterly audit for metric definitions, label integrity, cost-basis consistency, anti-gaming controls, and reproducibility sampling. Quarterly cadence is appropriate because metric drift can accumulate slowly and become visible only after several reporting cycles.

Anti-gaming rules for coding-agent productivity metrics

• Count attempts, not only wins: every rejected, retried, paused, and inconclusive task must remain in the denominator for cost and workload analysis.
• Lock metric definitions by reporting period: if a formula changes, annotate the dashboard and avoid comparing old and new periods without recalculation.
• Require evidence links for accepted outcomes: no task should count as accepted solely because an agent produced a plausible explanation or a polished diff.
• Separate task slicing from productivity: a team should not improve its success rate by breaking work into artificially tiny tasks that no longer represent meaningful research progress.
• Audit intervention under-reporting: compare transcripts, code-review comments, and task histories against intervention logs to detect missing steering events.
• Disallow self-approval for disputed outputs: contested outcomes need an independent reviewer or documented escalation path.
• Track paused work as control evidence: a pause gate is not a failed productivity event when it prevents scope drift, unsafe action, or non-reproducible results from advancing.

Conclusion

The practical lesson from OpenAI’s research-acceleration report is not that every organization should copy OpenAI’s internal ratios. The lesson is that coding-agent research work needs explicit definitions, supervised task boundaries, human-intervention logs, accepted-outcome evidence, cost normalization, anomaly detection, and pause gates before leaders can make credible productivity claims.

Use these final prompts as the operating layer above the earlier measurement prompts. The dashboard shows what is happening, anomaly detection challenges surprising trends, thresholds preserve useful human steering, failed-task review improves the system, pause gates protect the research process, executive reports preserve uncertainty, and the audit determines whether the measurement system is decision-grade.

Useful Links

• OpenAI: Research acceleration — a view inside OpenAI