Inside A Top Engineering Org: How They Shipped Full-Stack App Using AI Coding Agents

The 11-Week Sprint That Replaced a 9-Month Roadmap

In the first quarter of 2026, a 14-person engineering team at a Series C fintech startup accomplished what many would consider impossible: shipping a fully production-ready, customer-facing full-stack application in only 11 weeks. The original plan, devised in late 2025, forecasted a 9-month timeline with a larger 22-engineer team. Yet, through innovative use of AI coding agents and workflow restructuring, they defied expectations.

The application was complex and robust, incorporating a Next.js 15 frontend with React Server Components, a Go 1.23 backend spread across eight microservices totaling approximately 38,000 lines of code, a Postgres 16 database enhanced with pgvector for semantic search, and a full Stripe billing integration supporting subscriptions, metered billing, and tax calculations. The product also featured SOC 2-compliant audit logging, structured using OpenTelemetry and visualized through Grafana dashboards. A comprehensive Playwright end-to-end test suite with 184 scenarios ensured high reliability.

Most remarkably, the team experienced fewer production incidents in the first 60 days post-launch—only three compared to 17 in their prior release. Their post-launch code review backlog remained consistently low, never exceeding eight open pull requests. This demonstrated that AI coding agents, when integrated thoughtfully into a mature engineering culture, can accelerate delivery without compromising quality.

Why This Case Matters

Many AI-assisted development case studies falter under scrutiny. They often showcase limited, trivial projects or rely heavily on senior engineers to constantly vet AI output. This team’s success was different: it reflected a mature, production-grade deployment of AI agents with a full spectrum of engineering roles—4 senior, 6 mid-level, and 4 junior engineers—with juniors contributing meaningful agent-driven work by week four.

The CTO’s guiding principle was clear and uncompromising: “If we can’t trust the agent’s output enough to merge it with a single human review, we’re not gaining leverage — we’re just doing pair programming with extra overhead.” This mindset shaped every aspect of their AI workflow and tooling.

Full-Stack Scope Delivered

• Next.js 15 App Router frontend with 312 React Server Components and 47 routes
• Go 1.23 backend comprising 8 microservices with ~38K lines of code
• Postgres 16 database using pgvector for semantic search over transaction history
• Temporal workflows orchestrating asynchronous reconciliation and webhook retries
• Stripe integration supporting subscriptions, metered billing, tax, and 14 webhook handlers
• SOC 2-aligned audit trail with OpenTelemetry instrumentation and Grafana monitoring
• Comprehensive Playwright end-to-end testing suite with 184 scenarios and a 91% pass rate gating every PR

This was not a simple prototype or a wrapped API; it was a production-grade, regulated fintech product with real auditors scrutinizing every line.

The Agent Stack: Which Models Did What, and Why

Instead of relying on a single AI model to handle all coding tasks, the team deployed a specialized multi-agent stack. Each model was assigned to roles that leveraged its strengths, context length, pricing, and performance benchmarks. This strategic allocation was key to maintaining quality while controlling costs.

Model Assignment Matrix

Pricing verified against official docs from OpenAI, Anthropic, and OpenRouter as of April 2026.

The Two-Model Adversarial Review Workflow

The team’s most impactful innovation was a two-model adversarial review process: GPT-5.2-codex generated code and pull requests, then Claude Opus 4.7 performed an adversarial review before any human involvement. The reviewer agent was prompted to actively seek bugs, especially focusing on authentication, race conditions, and input validation.

Between weeks 5 and 7, the reviewer agent flagged 23 issues across 87 PRs. Humans confirmed 19 as real defects, 3 were stylistic, and only 1 was a false positive, representing an 86% precision rate. The cost per review was approximately $0.40, which the team calculated to be 30 times more cost-effective than a senior engineer finding a single auth bug.

For detailed engineering patterns, see our companion article Inside A YC Startup: How They Shipped Full-Stack App Using AI Coding Agents.

Why Multiple Models Instead of One?

Initially, the team tried using Claude Opus 4.7 exclusively for all tasks, benefiting from high output quality but hitting a prohibitive cost of $14,000 in two weeks. Cheaper models like GPT-5.4-mini proved effective for mechanical tasks but lacked nuanced reasoning.

GPT-5.5, released in April 2026, was reserved for design documents and incident retrospectives where its advanced reasoning justified its $5/$30 per million token cost, but GPT-5.2-codex remained the workhorse for code generation due to its specialization and cost-efficiency.

The Repository Scaffolding That Made Agents Productive

AI models don’t inherently understand your codebase. The team dedicated roughly 40% of their initial two weeks to preparing the repository and workflow scaffolding to maximize agent productivity. This investment separated them from teams struggling with mediocre AI output.

The AGENTS.md Convention

At the root of their approach was the adoption of AGENTS.md files placed at the root and in every significant package directory. These 200–600 word markdown files provide agents with structured, context-rich guidance including:

• Package purpose and responsibilities
• Public APIs and contracts
• Critical invariants and invariability warnings
• Testing conventions and commands
• Explicit “do not modify” files and areas
• Links to related packages and documentation

# AGENTS.md — billing-service

## Purpose
Handles all Stripe webhook ingestion, subscription state, 
and metered usage rollups. Source of truth for customer billing state.

## Public API
- gRPC: BillingService (see proto/billing/v1/billing.proto)
- REST: /api/v1/billing/* (gateway-exposed)

## Critical invariants
- NEVER mutate subscription state outside a Temporal workflow.
- ALL Stripe webhook handlers MUST be idempotent on stripe_event_id.
- Usage records are append-only. Corrections via reversal entries.

## Testing
- Unit: go test ./... (must pass)
- Integration: docker-compose -f test/docker-compose.yml up
- Webhook replay fixtures in test/fixtures/stripe/

## Do not modify without architect review
- internal/state/transitions.go (state machine)
- internal/idempotency/ (correctness-critical)

## Related context
- See ../audit-service/AGENTS.md for audit logging contract
- See ../../docs/billing-state-machine.md for state diagram

By automatically prefixing every prompt involving billing service code with this file’s content, agents avoided suggesting risky changes to critical files like transitions.go after week 3. This led to zero production idempotency bugs in Stripe webhook handling, a major improvement over prior releases.

Custom Tooling Surface

The team built a bespoke MCP (Model Context Protocol) server exposing 14 carefully designed tools tailored to their workflow:

1. repo.search_semantic: Vector embedding semantic code search via pgvector
2. repo.grep: Ripgrep wrapper returning up to 50 matches with context
3. repo.read_file: File reads with line range support
4. repo.edit_file: Applies unified diffs, rejecting conflicts
5. tests.run: Runs tests on changed files, outputs structured results
6. tests.run_e2e: Runs Playwright end-to-end tests on PR finalization
7. db.explain: Executes EXPLAIN ANALYZE on staging DB clones
8. db.migrate_dry_run: Validates schema migrations against snapshots
9. git.create_branch, git.commit, git.open_pr
10. review.request_second_opinion: Triggers cross-model adversarial review
11. observability.query_logs: Grafana Loki log queries
12. observability.query_metrics: Prometheus metrics queries
13. docs.fetch: Retrieves internal documentation by ID with allowlists

Importantly, the team deliberately excluded a shell.execute tool to avoid granting agents arbitrary command execution. This structured, auditable approach aligned with the company’s security posture and audit requirements.

See our detailed engineering trade-offs analysis in Inside Anthropic’s Claude Code Postmortem.

Prompt Caching: Cutting Costs

Both OpenAI and Anthropic support prompt caching that discounts repeated token inputs by 50–90%. The team structured their prompts with large, stable prefixes (~18K tokens) including system prompts, AGENTS.md contents, and tool definitions, followed by dynamic task suffixes.

With ~90% cache hit rates, their effective input token cost dropped from $1.25/M to about $0.20/M for GPT-5.2-codex. Total API spend for the 11-week project was $47,300 — roughly equivalent to adding 6–8 engineers’ output for the cost of slightly more than one.

The Daily Workflow: What Engineers Actually Did All Day

The workflow reshaped how engineers spent their time. Instead of writing code by hand for most of the day, their focus shifted towards specification, orchestration, review, and observability. Keyboard time in editors like vim dropped by approximately 70%.

Morning Kickoff: The Spec Session

Each engineer began their day with a 30–60 minute interactive spec session with GPT-5.5 or Claude Opus 4.7. This process refined feature tickets into precise, executable specifications, recorded as markdown files checked into the repo under specs/. A typical spec was 400–800 words, detailing:

• User-facing changes
• Affected files and predicted impact
• API contract changes
• Required test scenarios
• Rollback plans
• Non-obvious constraints

The team’s mantra was: “If you can’t write the spec, the agent shouldn’t write the code.” This prevented engineers from throwing vague requests at AI and receiving unusable outputs.

Code Generation Phase

With a completed spec, engineers used an internal CLI tool named swe that:

1. Loads the spec, repo context, and relevant AGENTS.md files
2. Initiates a GPT-5.2-codex session via MCP tools with up to 40 tool-call rounds
3. Caps spending at $3 per task
4. Streams progress live to the engineer’s terminal
5. Runs the test suite automatically and reports results
6. If tests pass, triggers a Claude Opus 4.7 adversarial review
7. If review passes, opens a PR with agent and reviewer notes

Engineers typically ran 4–8 such sessions concurrently across multiple worktrees. The bottleneck became human review, not code generation speed.

Managing Review Burden

At peak, the team generated ~40 PRs daily, averaging nearly 3 PRs per engineer. They managed this load with three core rules:

• Limit PR size: No PR over 400 lines without pre-approval; large specs were split.
• Use reviewer agent notes: Human reviewers prioritized flagged sections in PR descriptions.
• Explicit blast radius: Each PR included a description of impacted services, tables, and user surfaces.

Median time-to-merge for AI-generated PRs was 47 minutes, compared to 6.2 hours for human-written PRs before restructuring. This speed came from smaller, well-specified changes with automated adversarial review reducing manual effort.

Areas Reserved for Humans

Despite AI capabilities, some critical areas remained human-only:

1. Authentication and session management services, including a 2,400 LOC auth-service written exclusively by senior engineers
2. Destructive schema migrations like column drops or renames, requiring human-authored migrations and senior review
3. Cryptographic code involving key material, signing, or token validation

These choices were driven by risk management and audit accountability rather than AI ability.

For a broader view on AI adoption workflows, see How Engineering Teams Are Adopting AI Desktop Agents in 2026.

What Broke, and the Fixes That Stuck

No ambitious project is without setbacks. This team’s transparent recounting of failures offers invaluable lessons.

Week 2: Schema Drift Incident

Two engineers independently added a last_seen_at column migration to the users table, each successfully tested locally. When merged sequentially, the second migration failed in staging due to the column’s preexistence.

The solution was infrastructure-based: a “schema lease” service that the db.migrate_dry_run tool consulted to acquire exclusive migration leases. This serialized conflicting migrations and prevented race conditions.

Week 4: Infinite Tool-Call Loop

A GPT-5.2-codex debugging session entered a repetitive cycle of running tests, toggling edits, and reverting changes, exhausting its $3 budget in 12 minutes without progress.

The fix was a mandatory “what I learned” one-sentence summary before every tool call. If the same summary repeated thrice, the orchestrator halted the session and required human intervention. This forced agent self-awareness and eliminated infinite loops.

Week 6: Overconfident Refactor

GPT-5.1-codex-max produced a 2,800-line refactor touching 47 files in billing service. Tests passed and Claude Opus 4.7 approved the PR, which was merged. However, production saw a 400% spike in Stripe webhook retries due to silent error classification changes.

The team responded by:

• Requiring a third “skeptic” agent (Gemini 3.1 Pro or GPT-5.5) to review large PRs under adversarial assumptions
• Adding snapshot tests pinned to recorded Stripe events, immutable to agents without senior approval
• Mandating blast-radius declarations and auto-paging on-call engineers for billing logic changes

Week 9: Hallucinated API Usage

An agent used a deprecated query format for stripe.PaymentIntent.SearchAsync. Tests passed due to lax mocks, but the API call failed in production.

The fix was integrating a “real API contract” layer running against Stripe’s test mode in CI for critical endpoints, costing $1.20 per PR but catching API drift that static analysis missed.

The Trade-offs Nobody Talks About

Despite impressive metrics, the team acknowledged hidden costs and challenges.

Junior Engineer Skill Development

While juniors shipped features faster, their depth of system understanding and debugging skills lagged. During a staging incident in week 8, two juniors struggled to debug without agent help, a skill they would have mastered after 6 months in traditional workflows.

To address this, the org instituted “manual Wednesdays” where juniors pair with seniors on agent-free tasks to build independent debugging muscles.

Scaling Limits and Organizational Structure

The workflow functioned well with 14 engineers and ~38K LOC backend. However, scaling beyond 25 engineers would require reorganizing into smaller pods managing agent workflows with clear ownership boundaries.

Vendor Lock-in and Model Churn

Dependence on four AI vendors introduced churn. When GPT-5.5 launched, the team spent a week retraining and evaluating to decide whether to switch models, causing overhead despite specialization advantages favoring GPT-5.2-codex.

Interview Process Disruption

Their traditional interview loop focused on code-writing skills, now obsolete as candidates produce passable code using agents. The new process emphasizes spec-writing, code review under time constraints, and trade-off articulation, temporarily reducing hiring velocity by 50%.

Summary Comparison: Pre-AI vs Agent-Driven Workflows