How OpenAI Tripled GPT-5.6 Scores on ARC-AGI-3 with Two Simple API Settings: A Technical Deep Dive

The Benchmark That Separates True Reasoning from Pattern Matching

When OpenAI published a blog post detailing how two API configuration settings tripled GPT-5.6’s performance on ARC-AGI-3, the AI research community took notice immediately. This wasn’t a new model release, a new training run, or a fundamental architectural change. Two toggles. That’s all it took to move the needle from roughly 20% accuracy to over 60% on one of the most demanding reasoning benchmarks in existence. The implications of this discovery reach far beyond benchmark leaderboards — they fundamentally change how developers should be deploying frontier AI models today.

Understanding why this matters requires understanding what ARC-AGI-3 actually tests, why it has become the gold standard for measuring genuine machine intelligence, and why the specific capabilities unlocked by these settings speak directly to the deepest open questions in AI development. This article dissects the technical mechanics of both settings, examines the benchmark data in detail, and provides concrete implementation guidance for developers who want to extract maximum reasoning performance from the OpenAI API right now.

ARC-AGI-3: Why This Benchmark Is Different From Everything Else

The Abstraction and Reasoning Corpus, originally created by François Chollet at Google Brain and now stewarded by the ARC Prize Foundation, has a fundamentally different design philosophy from virtually every other AI benchmark. Where benchmarks like MMLU, HellaSwag, or even MATH measure performance on tasks drawn from a fixed distribution that models can be trained on, ARC was explicitly designed to be resistant to memorization. Each task presents a novel visual puzzle that requires genuine abstract reasoning — the kind of flexible, generalizable intelligence that humans apply effortlessly to new situations.

The Core Design Philosophy of ARC

A typical ARC task presents a small number of input-output grid pairs as demonstrations. The model must observe the transformation pattern — which might involve spatial reasoning, color logic, symmetry operations, counting, topology, or combinations of these — and then apply that pattern to a new input grid to produce the correct output. Crucially, each task uses a unique transformation rule. There is no training distribution of “ARC-style puzzles” that a model can memorize because the evaluation set contains tasks specifically designed to be outside any training distribution.

This design makes ARC one of the most honest measures of what researchers call fluid intelligence — the ability to reason about genuinely novel problems using core cognitive primitives. When a model scores high on ARC, it cannot be dismissed as retrieval from training data. It has to have actually reasoned through the problem. This is precisely why achieving human-level performance on ARC (roughly 85% for average adults) remains one of the most meaningful open challenges in AI, and why the ARC Prize Foundation has offered significant cash prizes for solutions that match human performance.

What Changed in ARC-AGI-3

ARC-AGI-3 represents a substantial difficulty increase over its predecessors. The ARC-AGI-1 benchmark saw models like GPT-4o struggle to reach 5% accuracy before specialized techniques were applied. ARC-AGI-2 raised the difficulty further and was released as a public challenge that humbled even the most capable frontier models. ARC-AGI-3 continues this escalation with several specific enhancements designed to close loopholes that allowed previous models to partially game earlier versions.

The third iteration introduces tasks with longer transformation chains, requiring models to track multiple intermediate states simultaneously. It also increases the complexity of spatial reasoning requirements, adding tasks that involve recursive patterns and multi-step logical conditionals that depend on prior steps in the grid transformation. Perhaps most importantly, ARC-AGI-3 evaluates whether models can maintain coherent reasoning across extended contexts — not just whether they can identify a transformation rule, but whether they can apply it consistently without losing track of state information accumulated during earlier analysis.

This last property — coherent reasoning across extended context — is precisely where the two API settings that OpenAI identified become critically important. The benchmark is, in a sense, a direct test of the very capabilities those settings are designed to preserve.

The Two Settings That Changed Everything

OpenAI’s blog post identified two specific API parameters that, when enabled together, produced the dramatic performance improvement on ARC-AGI-3. These are reasoning retention and reasoning compaction. While these parameter names may be updated or branded differently in the live API documentation as OpenAI refines its developer-facing language, the underlying mechanisms they control are technically distinct and worth understanding at a deep level. Developers using the OpenAI API for any complex multi-step reasoning task should treat this finding as immediately actionable intelligence.

Reasoning Retention: Preserving the Thinking Trail

The reasoning retention setting controls whether the model’s internal chain-of-thought reasoning tokens are preserved and carried forward in the context when making subsequent calls or when processing multi-part problems. To understand why this matters, it helps to understand what happens by default when a model processes a complex reasoning task.

In standard operation, when a model like GPT-5.6 (which uses OpenAI’s o-series reasoning architecture) works through a problem, it generates what are sometimes called “thinking tokens” — an extended internal monologue in which it works through intermediate steps, considers alternatives, and builds toward a conclusion. These tokens are not included in the final response visible to the user. They exist as scratchpad computation. By default, these tokens are also not included in the context when the conversation continues, because doing so would rapidly consume context window space and increase inference costs.

With reasoning retention enabled, the model’s intermediate reasoning states are preserved and made available for subsequent steps. On a task like ARC-AGI-3, this is enormously consequential. A complex grid transformation problem might require the model to: first identify candidate rules from the demonstration pairs, then test those candidates against each demonstration, then refine its hypothesis, then verify the refined hypothesis, and finally apply it to the test input. Without reasoning retention, each of these steps starts from scratch with only the compressed final outputs of previous steps. With reasoning retention, the model can build on its actual reasoning trail, accessing the nuanced intermediate conclusions it developed during earlier analysis rather than just the summarized endpoint.

Think of it as the difference between solving a complex math problem where you’re allowed to keep your scratch work visible versus one where you must memorize every intermediate result in your head before writing down only the final answer. The underlying cognitive capability is the same, but the working memory constraint changes what’s actually achievable. OpenAI o3 reasoning model technical breakdown

The Technical Mechanism: How Reasoning Tokens Accumulate Value

Reasoning retention works because chain-of-thought reasoning is not merely a formality — it is computationally doing real work. Research from papers including “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” and subsequent work on process reward models has established that the quality of intermediate reasoning steps directly predicts final answer accuracy on complex tasks. When those intermediate steps are discarded before subsequent processing, a model working through a multi-stage problem must essentially re-derive insights it already computed.

For ARC-AGI-3 specifically, the value of reasoning retention compounds across stages. The model’s initial analysis of the demonstration pairs generates specific hypotheses about transformation rules — hypotheses that often contain subtle qualifications, exceptions, and conditional branches that get compressed or lost when only the final summary passes forward. By preserving these, the model can apply more faithful and nuanced rule applications when generating its answer, catching edge cases it identified during analysis but might otherwise forget to account for.

Reasoning Compaction: Making Extended Reasoning Practical

The obvious objection to reasoning retention is the context window cost. If you preserve every thinking token from every reasoning step, you can rapidly exhaust even the largest available context windows on complex tasks. This is where reasoning compaction becomes essential — it is not merely an optimization, but the mechanism that makes reasoning retention practical at scale.

Reasoning compaction applies intelligent summarization to retained reasoning content. Rather than storing the full verbatim stream of thinking tokens, it identifies and preserves the most informationally dense segments — key hypotheses formed, critical logical deductions made, specific observations noted — while compressing or discarding the more exploratory, meandering portions of the reasoning trail. The goal is to maintain the substance of the model’s developed understanding while dramatically reducing the token footprint of that preserved reasoning.

This is technically non-trivial because identifying which parts of a reasoning trace are informationally dense requires its own form of meta-reasoning. OpenAI’s implementation appears to use a learned compression approach where the model has been trained to identify which intermediate conclusions in its own reasoning are most load-bearing — most likely to be needed for subsequent steps — and preferentially preserve those. The result is something closer to how human working memory actually functions: we don’t remember every word we read while solving a problem, but we do retain key insights and intermediate conclusions at higher fidelity. GPT-5 reasoning capabilities and how they work

Why These Two Settings Must Work Together

OpenAI’s data showed that either setting alone produced modest improvements, but the combination produced the dramatic tripling of performance. This is not surprising once you understand the mechanics. Reasoning retention without compaction rapidly fills context windows, forcing the model to either truncate its reasoning history or work with a drastically reduced window for the actual problem content. Compaction without retention produces efficiently summarized outputs of reasoning steps that are then discarded anyway — leaving you with well-organized disposable notes. Together, they create a system where valuable reasoning is preserved in a form that’s sustainable over extended multi-step problems of the type that dominate ARC-AGI-3.

The Numbers: Before and After Benchmark Analysis

The performance data OpenAI published is striking in both the magnitude of the improvement and the specific pattern of tasks that saw the largest gains. Understanding these patterns reveals important truths about where current frontier models still struggle and what kinds of problems benefit most from extended, preserved reasoning chains.

Overall Performance Metrics

GPT-5.6 operating under default API settings achieved approximately 20% accuracy on ARC-AGI-3 in OpenAI’s internal evaluations. With both reasoning retention and compaction enabled, this climbed to approximately 63% — a roughly 3x improvement. For context, the previous state-of-the-art on ARC-AGI-3 with any method had not exceeded 40% for a single-pass approach. The 63% figure achieved with these settings represents a substantial advance, though it still leaves a meaningful gap to human-level performance (approximately 85%).

Task Category Breakdown

The improvement was not uniform across all ARC-AGI-3 task types. OpenAI’s analysis revealed that the largest gains came specifically from tasks requiring multi-step rule application — problems where the transformation could not be identified from a single observation but required synthesizing evidence across multiple demonstration pairs. These tasks saw accuracy improve from roughly 12% to over 70%, a nearly six-fold increase.

Tasks involving what researchers classify as “counting and arithmetic transformations” — where the model must track numerical relationships between grid elements — showed the second-largest gains, moving from approximately 18% to 55%. Purely spatial transformation tasks that could be solved through a single clear observation saw smaller improvements, from around 30% to 48%. This pattern makes intuitive sense: the reasoning preservation benefit is proportional to how much valuable intermediate reasoning work would otherwise be lost.

Comparison to Previous Best Methods

Before these settings were identified, the best-performing approaches on ARC-AGI benchmarks typically involved test-time compute scaling through majority voting (running the same problem many times and taking the most common answer), program synthesis (having the model write code to solve the transformation and then executing that code), or search-based approaches that explored many candidate solutions. These methods can be effective but add significant latency and computational cost.

The remarkable thing about the reasoning retention and compaction approach is that it achieves competitive or superior results through a single forward pass per problem, just with better-configured reasoning infrastructure. This makes it dramatically more practical for real-world deployment. A developer building an application that needs strong reasoning performance no longer faces a choice between quality and latency — they can get both by enabling the right API settings. Best practices for OpenAI API integration in production

Why This Result Matters Beyond ARC-AGI-3

The ARC-AGI-3 results are compelling on their own terms, but treating this finding as merely a benchmark optimization would be a significant underestimation of its implications. The capabilities being tested by ARC-AGI-3 — extended multi-step reasoning, maintenance of state across complex analysis, coherent application of discovered rules to novel instances — are capabilities that matter intensely across a wide range of real-world applications.

What This Reveals About Existing Model Capability

One of the most important implications of this finding is what it reveals about the gap between a model’s latent capability and its deployed performance. GPT-5.6 is the same underlying model whether these settings are enabled or not. The model weights are identical. The training is identical. What changed is the infrastructure around how that model’s reasoning is managed during inference. This means that the 20% baseline score was not an accurate picture of GPT-5.6’s actual reasoning capability — it was a picture of GPT-5.6’s reasoning capability under a suboptimal operational configuration.

This is a sobering observation for anyone who has been evaluating model capabilities based on API performance alone. If a configuration change can triple performance on a hard reasoning benchmark without changing the model at all, then how many other benchmark results are substantially understated due to suboptimal inference configuration? OpenAI’s finding suggests that “model capability” cannot be separated from “inference configuration” in any meaningful evaluation. The two are intertwined, and optimizing one is as important as improving the other.

Implications for Complex Real-World Tasks

The types of tasks that benefit from reasoning retention and compaction on ARC-AGI-3 map directly onto categories of high-value professional work. Consider a model helping a software engineer debug a complex distributed system failure: the reasoning trail for identifying the root cause might span analysis of dozens of log entries, consideration and elimination of multiple hypotheses, and identification of subtle timing dependencies. With reasoning retention, the model can build a coherent diagnostic narrative where each observation builds on previous deductions. Without it, each step starts from a compressed summary of previous work, losing the texture of the reasoning that makes accurate diagnosis possible.

Similarly, financial analysis tasks that require building up an understanding of a company’s situation from multiple data sources, legal reasoning tasks that involve applying precedent to complex factual scenarios, or scientific hypothesis generation tasks that require synthesizing evidence across multiple experiments all exhibit the same structural property: the quality of later reasoning depends critically on access to the nuanced content of earlier reasoning, not just its compressed conclusion.

The State Tracking Problem in Long-Horizon Tasks

ARC-AGI-3’s emphasis on state tracking — maintaining accurate representations of intermediate grid states across transformation steps — maps onto a well-documented challenge in LLM deployment known as the long-horizon planning problem. Models tasked with executing multi-step plans, managing complex workflows, or conducting extended analysis consistently degrade in performance as the number of required steps increases. Much of this degradation has been attributed to the model “losing track” of intermediate state information as context fills with less relevant content.

Reasoning compaction addresses exactly this problem by treating the model’s own reasoning as the most valuable context to preserve. Rather than allowing earlier analytical conclusions to be pushed out of effective context by later conversational turns, compaction actively maintains the core reasoning infrastructure while allowing surface-level conversational content to occupy less privileged positions. This is not just valuable for benchmark performance — it is a fundamental architecture improvement for any application that involves extended, stateful reasoning. AI agent frameworks and long-horizon planning

How to Enable These Settings in Your OpenAI API Calls

For developers ready to implement these settings immediately, here is a detailed technical guide to enabling reasoning retention and compaction in the OpenAI API. Note that the exact parameter names and structures are subject to change as OpenAI refines its API, so developers should always cross-reference with the current official documentation, but the following reflects the configuration described in OpenAI’s published findings.

Basic API Configuration

These settings are configured through the reasoning parameter object in your API request. For models in the o-series (including GPT-5.6, o3, and subsequent reasoning models), you pass a reasoning configuration object alongside your standard request parameters. Here is an example using the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.6",
    input=[
        {
            "role": "user",
            "content": "Analyze the following transformation pattern and apply it to the test case..."
        }
    ],
    reasoning={
        "effort": "high",
        "retention": True,
        "compaction": True,
        "summary": "detailed"
    },
    max_output_tokens=8000
)

print(response.output_text)

The retention boolean controls whether the model’s reasoning tokens are preserved across the problem context. The compaction boolean activates intelligent compression of retained reasoning. The effort parameter, set to "high", tells the model to invest more compute in its reasoning process — this is distinct from but complementary to retention and compaction, and setting it appropriately ensures the model generates the kind of extended reasoning that these settings are designed to preserve.

Managing Context and Costs

Enabling reasoning retention does increase token usage, even with compaction active. Developers should be aware of several practical implications. First, the effective context consumed by a reasoning-retained conversation grows faster than a standard conversation, because reasoning summaries are added to context alongside the actual conversation content. Second, the max_output_tokens parameter should be set generously when these settings are active — constraining output tokens too tightly can force the model to truncate its reasoning in ways that undermine the benefits of retention.

# For multi-turn conversations with reasoning retention
# Maintain the conversation history and include previous
# reasoning summaries in subsequent turns

conversation_history = []

def send_with_reasoning(user_message, history):
    history.append({
        "role": "user",
        "content": user_message
    })
    
    response = client.responses.create(
        model="gpt-5.6",
        input=history,
        reasoning={
            "effort": "high",
            "retention": True,
            "compaction": True,
        },
        max_output_tokens=10000
    )
    
    # Include the response with preserved reasoning
    # context in subsequent turns
    history.append({
        "role": "assistant",
        "content": response.output_text,
        "reasoning_summary": response.reasoning.summary
    })
    
    return response.output_text, history

When to Use Each Setting

Not every use case benefits equally from both settings, and there are legitimate reasons to enable them selectively. The following table provides practical guidance on configuration choices based on task characteristics:

Cost-Benefit Optimization Strategies

Because enabling these settings increases token consumption, developers should implement strategies to use them judiciously. One effective approach is to use a routing layer that classifies incoming requests by complexity before deciding which configuration to use. A fast, lightweight classification call (using a smaller model) can determine whether a task has the multi-step, state-tracking character that benefits from reasoning retention, then route accordingly.

Another important optimization is to set appropriate reasoning.effort levels. The high effort setting allocates more internal compute to reasoning and generates richer reasoning traces worth preserving. For simpler tasks, even when you enable retention and compaction, using medium effort reduces the volume of reasoning content generated and preserved, keeping costs lower while still providing some benefit of maintained context.

Developers should also consider the problem decomposition opportunity: rather than throwing an entire complex problem at the model in a single turn with retention enabled, breaking the problem into coherent analytical stages and using retention across those stages can produce better results than either a single monolithic call or disconnected individual calls. This staged approach mirrors how the ARC-AGI-3 benefit actually manifests — the model builds understanding progressively, with each stage benefiting from preserved insights from all previous stages.

What This Means for the Broader AI Landscape

OpenAI’s finding represents more than a useful API tip. It opens a set of important questions about the nature of AI capability, how we measure it, and what “improving AI” actually means in a world where inference-time configuration can produce benchmark-scale improvements without any change to model weights.

The Inference-Time Optimization Revolution

The AI research community has been deeply engaged with inference-time compute scaling since OpenAI’s o1 model demonstrated that allocating more compute to the reasoning process itself — rather than just to training larger models — could produce dramatic capability improvements. The reasoning retention and compaction findings extend this insight in an important direction: it’s not just how much compute you allocate to reasoning at inference time, but how you manage that reasoning across complex multi-step tasks.

Anthropic has been exploring similar territory with Claude’s extended thinking mode, and Google DeepMind has published research on maintaining reasoning state in multi-step agent trajectories. The convergence of major labs on this set of techniques suggests that reasoning state management is emerging as one of the central technical challenges of the current AI development phase. Models are becoming increasingly capable of doing sophisticated reasoning in individual steps — the frontier is now in connecting those steps coherently across extended tasks.

Rethinking What “Model Capability” Means

The revelation that the same model can perform at 20% or 63% on the same benchmark depending on configuration should prompt a fundamental rethinking of how we discuss and compare AI capabilities. When we say “GPT-5.6 achieves X% on ARC-AGI-3,” we need to specify the inference configuration. The same is true of any benchmark result from any lab — if the reported numbers don’t specify reasoning settings, context management strategies, and inference configurations, they are underdetermined as capability claims.

This creates an important obligation for the AI benchmarking community to establish standardized evaluation protocols that either specify inference configurations precisely or report results across a range of configurations. The ARC Prize Foundation and other benchmark stewards will likely need to update their evaluation standards to address this. Otherwise, benchmark competitions become, in part, competitions over inference configuration optimization rather than pure model capability comparison.

Developer Responsibility and Capability Access

From a practical standpoint, this finding creates an interesting dynamic around developer access to model capability. The implication is that a significant amount of GPT-5.6’s capability has always been present but inaccessible to developers who did not know about these settings. This raises the question: what other configuration options are similarly high-leverage, and how should OpenAI communicate them to developers?

There’s a good argument that OpenAI should consider making reasoning retention and compaction defaults for complex tasks, using automatic complexity detection to determine when they’re beneficial. The current situation — where substantial capability improvements are available but require developers to know to enable specific parameters — creates an inequality between sophisticated developers who know about these settings and the broader developer community that doesn’t. Democratizing access to optimal inference configuration is as important as making the underlying models available. How to optimize ChatGPT API performance for enterprise applications

What This Implies About the Path to AGI

The ARC-AGI benchmark was created by François Chollet specifically as a test of general intelligence because he believed that genuine AGI would need to score near human level on it. GPT-5.6’s jump from 20% to 63% with configuration changes is significant progress, but the gap to human-level performance (~85%) remains substantial. More importantly, the nature of that remaining gap is revealing.

The tasks that still stump GPT-5.6 with optimal settings tend to be those requiring what Chollet calls “the highest level of abstraction” — problems where the transformation rule involves a concept or relationship that requires genuine conceptual innovation rather than recombination of known primitives. These are the tasks where even extended, well-preserved reasoning chains cannot compensate for a fundamental gap in the conceptual toolkit available to the model. Solving these tasks likely requires advances in model architecture and training, not just inference configuration optimization.

This delineation is actually clarifying for the field: it suggests that reasoning retention and compaction have largely addressed the “lost reasoning context” failure mode that was artificially suppressing performance, and what remains is a cleaner signal about where genuine architectural and training improvements are still needed. The remaining 22 percentage points to human performance on ARC-AGI-3 likely represent a harder and more fundamental challenge than the 43 points that were recovered through configuration optimization.

Practical Implications for Different Developer Categories

The impact of these findings varies significantly depending on how you’re using the OpenAI API. Here is a structured analysis of implications for major developer use cases.

Agentic Application Developers

For developers building AI agents — systems where the model must execute multi-step plans, interact with tools, process results, and adapt its approach based on what it learns — reasoning retention and compaction are immediately and highly relevant. The failure modes in current agentic systems often trace directly to the reasoning state degradation problem: agents that confidently pursue a strategy in step 3 that contradicts an important constraint they identified in step 1, because the detailed reasoning from step 1 is no longer accessible as rich context.

Enabling these settings in agentic frameworks should meaningfully improve plan coherence and reduce the frequency of logic contradictions across long action sequences. Developers should rebuild their agent evaluation frameworks to specifically test plan coherence across 10+ step tasks, both with and without these settings, to quantify the benefit in their specific use cases.

Scientific and Research Tool Developers

Applications that help researchers analyze data, synthesize literature, or generate hypotheses involve some of the most complex multi-step reasoning patterns in any professional domain. A literature synthesis task might involve: categorizing papers by methodology, identifying patterns across categories, noting contradictions between findings, generating explanatory hypotheses for those contradictions, and evaluating those hypotheses against additional evidence. Each stage of this process benefits enormously from preserved access to the insights generated in previous stages.

Scientific tool developers should consider task decomposition strategies that map naturally onto the staged reasoning architecture these settings enable. Rather than asking the model to “synthesize this literature,” explicitly structuring the task as a sequence of analytical stages — categorize, pattern-find, identify tensions, hypothesize, evaluate — and using reasoning retention across those stages should produce qualitatively better scientific reasoning output.

Software Engineering and Code Analysis Tools

Code analysis, debugging, and architecture review tasks have a property directly analogous to ARC-AGI-3: the model must maintain a coherent understanding of a complex state (the codebase) across multiple analytical operations (reading different files, tracing execution paths, identifying dependency relationships). Reasoning retention allows the model to build a genuine accumulated understanding of a codebase across a multi-file analysis session, rather than treating each file read as an independent operation that cannot fully leverage insights from previous reads.

For complex debugging tasks specifically — where the model must form and test hypotheses about root causes across multiple diagnostic steps — the benefit of preserved reasoning chains maps directly onto the ARC-AGI-3 multi-step rule identification scenario. The diagnostic reasoning accumulated across early steps becomes directly available for application in later steps, producing more accurate and confident root cause identification.

Legal and Financial Analysis Applications

High-stakes professional analysis applications in legal and financial domains often require exactly the kind of evidence-accumulation and hypothesis-refinement reasoning that reasoning retention is designed to support. A contract analysis task might require the model to identify relevant clauses, build a model of the contractual obligations, identify potential conflicts between clauses, and then evaluate specific scenarios against that model. A financial analysis task might require building a model of a company’s financial health from multiple statements, identifying risk factors, and evaluating their likelihood and impact.

For applications in these domains, the combination of higher accuracy and better reasoning traceability (the compacted reasoning summary can be inspected to understand how conclusions were reached) provides both quality and compliance benefits. Developers building applications for regulated industries should consider whether the preserved reasoning summaries can also serve as audit trails for AI-assisted decision support.

Technical Limitations and Honest Caveats

No technical finding comes without important caveats, and intellectual honesty requires addressing the limitations of what OpenAI has demonstrated.

Benchmark vs. Real-World Performance

ARC-AGI-3 is an excellent benchmark for measuring genuine abstract reasoning, but it has specific structural properties that may not generalize uniformly to all real-world tasks. The benchmark consists of short visual tasks with clean, objective correct answers. Real-world reasoning tasks often involve ambiguity, incomplete information, competing valid interpretations, and evaluation criteria that are not binary correct/incorrect. The magnitude of improvement seen on ARC-AGI-3 may overestimate improvements on more typical professional tasks, particularly those where the limiting factor is knowledge rather than multi-step reasoning.

Cost Scaling Concerns

While reasoning compaction substantially reduces the token overhead of retention compared to naive approaches, the combined settings still increase API costs. For high-volume, low-complexity use cases, the cost increase is unlikely to be justified by performance improvements. Developers should conduct careful cost-benefit analysis and consider tiered deployment strategies where these settings are reserved for problems that demonstrably benefit from them.

Model-Specific Results

OpenAI’s published results are specific to GPT-5.6. The magnitude of benefit from reasoning retention and compaction likely varies across models. Models with weaker baseline reasoning capabilities may see different improvement patterns; models that are not part of the o-series reasoning architecture may see different or smaller benefits. Developers using GPT-4o, GPT-4.1, or other non-reasoning-specialized models should not assume identical performance improvements from enabling these settings.

The Remaining 22% Gap

Even with optimal settings, GPT-5.6 scores approximately 63% on ARC-AGI-3 against a human baseline of ~85%. This remaining gap should not be minimized. The tasks in that gap represent genuine unsolved challenges in AI reasoning, and no amount of inference configuration optimization will close them without fundamental advances in model architecture and training. Developers building applications that depend on the highest levels of abstract reasoning should benchmark carefully rather than extrapolating from the ARC-AGI-3 headline improvement.

The Road Ahead: What Comes After Configuration Optimization

OpenAI’s discovery of these high-leverage configuration settings is best understood as the optimization of existing capability, not the creation of new capability. The next frontier of ARC-AGI performance improvement will require advances that go beyond inference configuration: more powerful reasoning architectures, training approaches that better develop fluid intelligence, and potentially novel approaches to how models represent and manipulate abstract structures.

The ARC Prize Foundation has consistently argued that solving ARC at human level requires something closer to the “program synthesis” approach — models that don’t just reason in language about transformations but can construct and execute actual algorithmic solutions. Hybrid approaches that combine the kind of language-based reasoning that retention and compaction enhance with the kind of executable computation that program synthesis provides may be the path to closing the remaining gap. Several research groups are actively exploring this direction, and it represents one of the most intellectually exciting frontiers in AI capability research today.

For the immediate future, what OpenAI has demonstrated is that the gap between a model’s evaluated benchmark performance and its actual capability can be substantial, and that closing this gap through careful inference configuration is an underexplored and high-value direction. Developers who internalize this lesson — who treat inference configuration as a first-class optimization target alongside prompt engineering and model selection — will extract substantially more value from frontier AI systems than those who rely on default settings and assume that benchmark numbers tell the complete story.

The tripling of ARC-AGI-3 performance through two API settings is not just a technical curiosity. It is evidence that the AI capability landscape is more malleable, more configuration-dependent, and frankly more optimistic than standard benchmark reporting suggests. The models that exist today, properly configured and deployed, are capable of reasoning at levels that default deployments significantly understate. The work of unlocking that capability — through settings like reasoning retention and compaction, through intelligent inference management, through thoughtful task decomposition — is some of the highest-leverage work available to the AI developer community right now.

The most important takeaway for developers is actionable: if you are using the OpenAI API for tasks that involve multi-step reasoning, extended analysis, or state tracking across a complex problem, you should test reasoning retention and compaction settings immediately. The performance improvement seen on ARC-AGI-3 is a strong signal that these settings address a real and pervasive limitation in how frontier models have historically been deployed. The capability improvement is sitting there, waiting to be accessed. The cost is a few additional lines of configuration code and a moderate increase in token usage. For the vast majority of complex reasoning applications, that tradeoff is clearly worth making.