How to Use the New GPT-5.5 Instant Update for E-Commerce: Building AI-Powered Shopping Assistants with Product Recommendations, Constraint Handling, and Intent Recognition

Markos Symeonides

Harnessing OpenAI’s GPT-5.5 Instant Update for E-Commerce: Build Intelligent AI Shopping Assistants with Advanced Product Recommendations, Constraint Management & Intent Detection

Executive Summary:
This in-depth technical tutorial unpacks the transformative June 24, 2026 update to OpenAI’s GPT-5.5 Instant model, spotlighting its game-changing applications in retail and e-commerce. You’ll receive a fully production-ready blueprint and Python/FastAPI implementation for creating AI-driven shopping assistants that excel in natural language intent recognition, multi-constraint handling, and sophisticated multi-turn conversational workflows.
Learn how to architect end-to-end solutions—including system prompt engineering, schema-driven structured outputs, error resilience with retries and circuit breakers, and hybrid deterministic-model orchestration. The guide thoroughly covers integration with vector search engines, product data validation, personalization enrichment, and operational essentials like deployment, observability, SLOs, and A/B testing for scalable, robust AI commerce experiences.

Introduction: The Breakthrough GPT-5.5 Instant E-Commerce Update & Retail Impact

On June 24, 2026, OpenAI delivered a pivotal upgrade to GPT-5.5 Instant tailored for demanding e-commerce environments. This update enhances:

  • Intent Recognition Precision: Sharper understanding of nuanced and compound user queries.
  • Robust Constraint Handling: Reliable enforcement of complex, multi-factor constraints such as budgets, sizing, and brand exclusions.
  • Multi-Turn Workflow Orchestration: Managing multi-step shopping dialogues—from product search and filtering to checkout intent—within single coherent conversations.

The model’s internal reasoning and prompt engine optimizations drastically cut down brittle heuristics and ad hoc backend logic. While a well-instrumented orchestration layer is still mandatory to ensure data accuracy and transaction integrity, the cognitive burden on engineering teams mapping intent to operational requests is significantly lightened.

Target Audience & Objectives

This comprehensive guide targets engineering teams, technical product managers, and ML engineers integrating GPT-5.5 Instant AI assistants into large-scale retail systems. Our mission: to equip you with practical architectural insights, prompt design patterns, and battle-tested code that you can customize for your unique product catalog, personalization engines, and production requirements.

Project Overview: What You’ll Build

By following this walkthrough, you will develop:

  • A scalable FastAPI-based API for a GPT-5.5 Instant powered e-commerce shopping assistant.
  • Robust system prompts paired with strict JSON schema definitions for deterministic output parsing.
  • A hybrid constraint solving pipeline blending GPT’s enhanced capabilities with deterministic filters and linear optimization methods suited for bundle and multi-item requests.
  • Reliable error handling featuring exponential backoff, retries, and circuit breaker mechanisms for stable AI service calls.
  • An actionable deployment blueprint covering Docker, Kubernetes manifests, monitoring integrations, and SLOs focused on latency and output correctness.

Prerequisites & Recommended Familiarity

  • Proficiency in Python 3.10+ and FastAPI framework.
  • Understanding of REST API fundamentals, JSON schemas, and containerization (Docker/Kubernetes).
  • Familiarity with vector databases like Pinecone, Weaviate, or Milvus (optional but advantageous) for semantic product retrieval tasks.
  • Access to a GPT-5.5 Instant API key with proper environment variable setup.

Section 1: Decoding GPT-5.5 Instant’s Advanced E-Commerce Capabilities

1.1 Enhanced Intent Recognition for Complex Shopping Queries

The latest GPT-5.5 Instant model iteration integrates a refined semantic parser and pattern detector that excels at:

  • Disentangling compound intents: Handling multifaceted requests such as “Find sneakers under $120 that ship to Berlin in 2 days.”
  • Extracting granular constraints: Detecting budgets, material preferences, style attributes, and urgency signals (e.g., “ASAP”, “by Friday”).
  • Incorporating conversation context: Remembering customer sizes, previous brand disqualifications, and evolving preferences across dialogue turns for hyper-personalized interactions.

By leveraging these capabilities, your applications can offload rigorous input parsing to the model while ensuring structured, normalized output validation downstream.

1.2 Advanced Multi-Constraint Problem Solving with Schema-Driven Outputs

This update notably improves the model’s internal reasoning to satisfy overlapping constraints such as price ceilings, sizing, color preferences, and shipping logistics simultaneously. Highlights include:

  • Reduced hallucination: The model enforces constraint adherence more faithfully, decreasing incorrect candidate recommendations.
  • Strict JSON output adherence: System prompts guide the model to produce machine-readable, schema-compliant JSON that can be reliably parsed.
  • Stepwise reasoning: The model can generate candidate listings, apply filters iteratively, and present comparative trade-offs—all in structured formats.

Important: AI reasoning supplements but does not replace deterministic validation. We strongly advocate hybrid architectures combining model inference with backend constraint enforcement to guarantee business-critical accuracy—demonstrated in upcoming sections.

1.3 Seamless Multi-Turn Shopping Conversation Orchestration

GPT-5.5 Instant empowers developers to build fluid, multistep conversational shopping experiences that guide users through:

  1. Product discovery and shortlist creation.
  2. Variant and size availability checks.
  3. Shipping options and promotion coordination.
  4. Reservation and checkout intent confirmation.

Key best practices include:

  • Persist conversation context with performant memory stores (e.g., Redis) to enable coherent follow-ups without token overload.
  • Utilize clear phase-transition signals via structured next_action tokens for orchestrator-triggered backend integrations.
  • Enforce consistent structured response schemas on every turn to facilitate automated downstream processing.

1.4 Illustrative Examples of Structured Model Outputs

Example 1: Multi-Constraint Product Search Request

{
  "user_request": "Show me 3 casual shirts in medium under $70, prefer cotton, no prints, ship next-day to San Francisco",
  "parsed_intent": {
    "type": "product_search",
    "categories": ["shirts", "casual"],
    "size": "M",
    "price_max": 70,
    "material": ["cotton"],
    "exclude": ["printed"],
    "shipping_speed": "next-day",
    "shipping_zip": "94103"
  },
  "top_candidates": [
    {"product_id": "SKU-1001", "score": 0.93, "price": 65},
    {"product_id": "SKU-3098", "score": 0.90, "price": 69},
    {"product_id": "SKU-8074", "score": 0.88, "price": 62}
  ],
  "next_action": "confirm_selection"
}

Example 2: Bundle Optimization Under Budget Constraints

{
  "user_request": "Put together a travel outfit for Europe under $500, include shoes, jacket, and one layering piece. Prefer sustainable brands.",
  "parsed_intent": { "type": "bundle_request", "budget": 500, "required_items": ["shoes","jacket","layering"], "brand_quality": "sustainable" },
  "solution": {
    "items": [
      {"product_id": "S-345", "price": 160, "notes": "water-resistant jacket"},
      {"product_id": "SH-912", "price": 140, "notes": "lightweight travel shoes"},
      {"product_id": "L-223", "price": 95, "notes": "merino cardigan"}
    ],
    "total_price": 395,
    "budget_remaining": 105
  },
  "next_action": "offer_checkout_or_more_options"
}

These structured outputs facilitate rigorous validation, intuitive filtering, and seamless orchestration. We will implement this full processing pipeline in Sections 2 and 3.

Section 2: Designing a Scalable AI-Powered Shopping Assistant Architecture

2.1 Core System Components and Responsibilities

Creating a resilient GPT-5.5 Instant-backed shopping assistant requires thoughtfully orchestrating these key components:

  • Client Frontend: Mobile/web interface capturing natural language inputs and rendering candidate results; performs lightweight local data validation and telemetry.
  • API Gateway & FastAPI Service: Central orchestrator managing session contexts, product interactions, model API calls, and producing actionable structured responses.
  • Product Data Layer: Robust catalog database and microservices handling inventory, pricing, and promotional logic with authoritative data.
  • Vector Store & Semantic Retriever: Enables semantic matching on open-ended queries using embedding similarity (Pinecone, Weaviate, Milvus).
  • Context Memory Store: Low-latency key-value system (e.g., Redis) to manage short-term session state and conversational context.
  • Personalization & User Profile Store: Long-term user data used for personalized ranking and filtering, with privacy safeguards.
  • Orchestrator & Deterministic Tools: Enforces hard constraints, manages inventory checks, pricing filters, bundle optimization solvers, and executes transactional actions.
  • GPT-5.5 Instant Model API: Handles natural language understanding, intent parsing, candidate filtering suggestions, and dialogue continuation.
  • Observability & Monitoring: Comprehensive telemetry pipeline for metrics, tracing, logging using Prometheus, Grafana, and distributed tracing frameworks.

2.2 High-Level Data Flow and Key Processing Steps

  1. User Input: Frontend collects user utterance and metadata, forwarding it to the API service.
  2. Session Context Retrieval: API fetches session short-term memory plus long-term user profile.
  3. Semantic Candidate Retrieval: Query vector index for semantically related products if applicable.
  4. Model Invocation: Submit conversation history, user intent, and candidate metadata to GPT-5.5 Instant with structured system prompts requesting JSON outputs.
  5. Post-Model Validation: Deterministic rule checks confirm constraints on price, stock, size, filtering out invalid candidates.
  6. Business Logic Application: Promotions, loyalty discounts, and bundle computations applied deterministically or via optimization solvers.
  7. Response Dispatch: Validated structured results returned to frontend for presentation and user interaction.
  8. Telemetry Logging & Feedback Loop: Record requests, model outputs, and downstream user behavior for continuous improvement and prompt tuning.

2.3 Conversation Context Management Best Practices

  • Short-Term Memory: Retain the last 6-10 turns per session or summarized context entries to maintain dialogue coherence efficiently.
  • Context Summarization: Leverage model-assisted summarization to compress session history, preserving essential preferences while controlling token consumption.
  • Schema-Based State: Prefer structured context objects (e.g., last size, excluded brands) over raw textual history for token and latency optimization.
  • Personalization Integration: Inject user purchase history and preferences as structured JSON fields, respecting privacy policies.

2.4 Emphasizing Product Data Quality & Hygiene

  • Normalizing product attributes—consistent size standards, canonical color names, unified material taxonomies.
  • Ensuring inventory and price are always sourced from authoritative backend systems rather than cached or model outputs.
  • Passing succinct product metadata summaries to the model during calls; querying full details post-model for verification and fulfillment.

2.5 Hybrid Model & Deterministic Orchestration Pattern

The recommended approach blends GPT-5.5 Instant’s semantic parsing and candidate suggestion with carefully architected backend logic that enforces irreversible operations (inventory reservation, payment authorization).
For instance, the model may output an action token like reserve_item, but your backend must execute the transactional reserve logic securely and idempotently.
This strategy balances AI flexibility with operational robustness and compliance.

Additional Notes:

  • To reduce latency and token consumption, pass product embeddings or concise textual summaries rather than entire catalog data.
  • Always cross-verify model outputs against canonical system constraints and treat them as suggestions until system validation confirms.

For an in-depth study on semantic retrieval in e-commerce, review our engineering case study: How to Build a Research Assistant with Claude Code in 2026: Step-by-Step

Section 3: Practical Step-by-Step Code Implementation

This section delivers the end-to-end FastAPI microservice code integrating GPT-5.5 Instant, semantic retrieval, robust error handling, structured schema validation, and deterministic constraint enforcement.

3.1 Modular Project Layout

ai-shopping-assistant/
├─ app/
│  ├─ main.py                   # API entrypoint using FastAPI
│  ├─ api/
│  │  ├─ endpoints.py           # Define REST routes
│  ├─ services/
│  │  ├─ model_client.py        # GPT-5.5 Instant API wrapper with retries/circuit breaker
│  │  ├─ retriever.py           # Vector store semantic search
│  │  ├─ catalog.py             # Catalog checks & verification
│  │  ├─ constraint_solver.py   # Bundle optimization strategies (greedy/LP)
│  ├─ schemas/
│  │  ├─ models.py              # Pydantic data models for request & response validation
│  ├─ utils/
│  │  ├─ backoff.py             # Retry decorators
│  │  ├─ circuit_breaker.py     # Circuit breaker implementation
│  ├─ config.py                 # Configuration management
├─ tests/
│  ├─ test_constraint_solver.py
│  ├─ test_model_client.py
├─ Dockerfile
├─ k8s/
│  ├─ deployment.yaml
│  ├─ hpa.yaml
├─ requirements.txt

3.2 Environment & Dependencies Setup

Recommended Python dependencies (example):

# requirements.txt
fastapi==0.95.0
uvicorn[standard]==0.23.2
httpx==0.24.0
pydantic==2.5.0
tenacity==8.2.0
redis==4.5.5
numpy==1.27.0
scipy==1.11.3
python-or-tools==9.6.2537
prometheus-client==0.17.0
structlog==23.1.0

Note: Replace python-or-tools with pulp or other solvers per licensing and infrastructure needs. We leverage tenacity for retry logic and a compact circuit breaker implementation provided later.

3.3 Pydantic Data Schemas for Deterministic Parsing

# app/schemas/models.py
from pydantic import BaseModel, Field
from typing import List, Optional, Literal, Dict

class UserRequest(BaseModel):
    user_id: Optional[str]
    session_id: Optional[str]
    text: str
    locale: Optional[str] = "en-US"
    metadata: Optional[Dict[str, str]] = None

class ParsedIntent(BaseModel):
    type: Literal["product_search", "bundle_request", "checkout_intent", "clarify"] = "product_search"
    categories: Optional[List[str]] = None
    size: Optional[str] = None
    price_min: Optional[float] = None
    price_max: Optional[float] = None
    materials: Optional[List[str]] = None
    exclude: Optional[List[str]] = None
    shipping_speed: Optional[str] = None
    destination_zip: Optional[str] = None
    budget: Optional[float] = None
    required_items: Optional[List[str]] = None

    model_config = {"extra": "forbid"}


class Product(BaseModel):
    id: str
    title: str
    category: str
    price: float = Field(ge=0)
    sizes: List[str]
    materials: List[str]
    in_stock: bool
    shipping_days: int = Field(ge=0)

    model_config = {"extra": "forbid"}


class Recommendation(BaseModel):
    product_id: str
    reason: str
    score: float = Field(ge=0, le=1)

    model_config = {"extra": "forbid"}


class AssistantResponse(BaseModel):
    intent: ParsedIntent
    recommendations: List[Recommendation] = Field(default_factory=list)
    clarifying_question: Optional[str] = None
    notes: List[str] = Field(default_factory=list)

    model_config = {"extra": "forbid"}

3.4 Calling the Model with Structured Outputs

Keep the application models above separate from the schema sent to the model. Application models benefit from defaults, but OpenAI’s Structured Outputs guide requires every field in a model-facing schema to be required. Represent a logically optional value with a union that includes None, without assigning a default. Object schemas must also disable additional properties.

The native Pydantic helper keeps the Python type and generated schema together. The SDK supports client.responses.parse(..., text_format=ModelType) and exposes the parsed value through response.output_parsed. Set OPENAI_MODEL to a model identifier supported by your API project rather than embedding one in source code.

# app/llm/intent.py
import os
from typing import List, Literal

from openai import (
    APIConnectionError,
    APITimeoutError,
    InternalServerError,
    LengthFinishReasonError,
    OpenAI,
    RateLimitError,
)
from pydantic import BaseModel, ConfigDict, ValidationError

from app.schemas.models import ParsedIntent

MODEL = os.environ["OPENAI_MODEL"]
client = OpenAI(max_retries=0)

RETRYABLE_ERRORS = (
    APIConnectionError,
    APITimeoutError,
    InternalServerError,
    RateLimitError,
)


class IntentExtraction(BaseModel):
    model_config = ConfigDict(extra="forbid")

    type: Literal[
        "product_search",
        "bundle_request",
        "checkout_intent",
        "clarify",
    ]
    categories: List[str] | None
    size: str | None
    price_min: float | None
    price_max: float | None
    materials: List[str] | None
    exclude: List[str] | None
    shipping_speed: Literal["same-day", "next-day", "two-day"] | None
    destination_zip: str | None
    budget: float | None
    required_items: List[str] | None


SYSTEM_PROMPT = """
Extract shopping intent from the user's message.
Return only the fields defined by the response schema.
Use null when a value was not supplied.
Do not invent a budget, size, destination, or delivery deadline.
Use type "clarify" when the request is unrelated or cannot be parsed.
"""


def _contains_refusal(response: object) -> bool:
    for item in getattr(response, "output", []):
        for part in getattr(item, "content", []):
            if getattr(part, "type", None) == "refusal":
                return True
    return False


def _clarify_intent() -> ParsedIntent:
    return ParsedIntent(type="clarify")


def _validate_semantics(parsed: IntentExtraction) -> bool:
    if parsed.price_min is not None and parsed.price_min < 0:
        return False
    if parsed.price_max is not None and parsed.price_max < 0:
        return False
    if (
        parsed.price_min is not None
        and parsed.price_max is not None
        and parsed.price_min > parsed.price_max
    ):
        return False
    if parsed.budget is not None and parsed.budget < 0:
        return False
    return True


def extract_intent(user_text: str) -> ParsedIntent:
    try:
        response = client.responses.parse(
            model=MODEL,
            input=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": user_text},
            ],
            text_format=IntentExtraction,
        )
    except RETRYABLE_ERRORS:
        raise
    except (LengthFinishReasonError, ValidationError):
        return _clarify_intent()

    if getattr(response, "status", None) == "incomplete":
        return _clarify_intent()
    if _contains_refusal(response):
        return _clarify_intent()

    parsed = response.output_parsed
    if parsed is None or not _validate_semantics(parsed):
        return _clarify_intent()

    try:
        return ParsedIntent.model_validate(parsed.model_dump())
    except ValidationError:
        return _clarify_intent()

Refusal and incomplete-response handling must be part of normal control flow. A truncated structured response can fail while the SDK is parsing it, before application code can inspect the response status. Parsing failures, semantic validation failures, and application-model validation failures therefore become a clarify intent. Connection, timeout, rate-limit, and server failures remain eligible for the retry policy shown later.

Schema adherence does not guarantee that extracted values are correct. The semantic checks reject negative amounts and inverted price ranges before those values reach catalog logic. The schema also restricts shipping speed to labels understood by the deterministic filter, preventing an unknown label from being misreported as an empty catalog result.

Add domain-specific validation for normalized size labels, destination formats, and supported categories. Those checks depend on the retailer’s catalog and fulfillment rules rather than JSON shape.

3.5 Deterministic Constraint Filtering and Ranking

The model identifies likely intent; ordinary code decides whether a product satisfies it. Price, category, stock, size, material, exclusion, and delivery requirements are hard filters. This prevents natural-language output from bypassing an authoritative catalog rule.

# app/services/constraint_solver.py
from typing import List

from app.schemas.models import ParsedIntent, Product, Recommendation

SHIPPING_LIMITS = {
    "same-day": 0,
    "next-day": 1,
    "two-day": 2,
}


def _normalized(values: List[str] | None) -> set[str]:
    return {value.strip().casefold() for value in values or []}


def filter_products(
    products: List[Product],
    intent: ParsedIntent,
) -> List[Recommendation]:
    requested_materials = _normalized(intent.materials)
    excluded_terms = _normalized(intent.exclude)
    requested_categories = _normalized(intent.categories)
    requested_size = intent.size.strip().casefold() if intent.size else None

    shipping_limit = None
    if intent.shipping_speed:
        shipping_limit = SHIPPING_LIMITS.get(
            intent.shipping_speed.strip().casefold()
        )
        if shipping_limit is None:
            return []

    recommendations: List[Recommendation] = []

    for product in products:
        if not product.in_stock:
            continue
        if (
            requested_categories
            and product.category.strip().casefold() not in requested_categories
        ):
            continue
        if intent.price_min is not None and product.price < intent.price_min:
            continue
        if intent.price_max is not None and product.price > intent.price_max:
            continue

        product_sizes = _normalized(product.sizes)
        product_materials = _normalized(product.materials)
        searchable = " ".join(
            [product.title, product.category, *product.materials]
        ).casefold()

        if requested_size and requested_size not in product_sizes:
            continue
        if requested_materials and not requested_materials.issubset(
            product_materials
        ):
            continue
        if excluded_terms and any(term in searchable for term in excluded_terms):
            continue
        if shipping_limit is not None and product.shipping_days > shipping_limit:
            continue

        score = 0.50
        if intent.price_max is not None and intent.price_max > 0:
            price_headroom = max(
                0.0,
                1.0 - (product.price / intent.price_max),
            )
            score += 0.30 * price_headroom
        if shipping_limit is not None:
            divisor = max(shipping_limit, 1)
            delivery_headroom = max(
                0.0,
                (shipping_limit - product.shipping_days) / divisor,
            )
            score += 0.20 * delivery_headroom

        recommendations.append(
            Recommendation(
                product_id=product.id,
                reason=(
                    "Passes all hard constraints; ranked by price "
                    "and delivery headroom."
                ),
                score=min(score, 1.0),
            )
        )

    return sorted(
        recommendations,
        key=lambda item: (-item.score, item.product_id),
    )

Only eligible products reach the ranking stage. The score then uses two values that can differ among survivors: room below the maximum price and room inside the delivery deadline. It is a transparent ordering rule, not a claim about product quality or customer preference.

Regional pricing, promotion eligibility, current inventory, and delivery estimates should be checked against authoritative services immediately before presenting or reserving an item.

3.6 Exposing the Assistant with FastAPI

The endpoint connects extraction and filtering. CATALOG is an empty in-memory collection so the module starts without fabricated product records. Replace it through dependency injection or an adapter to the catalog service.

For compactness, this tutorial places the route in app/main.py and the intent logic in app/llm/intent.py. The layout’s app/api/endpoints.py, app/config.py, catalog.py, and retriever.py remain extension points for route separation, centralized settings, authoritative catalog access, and semantic retrieval.

# app/main.py
from typing import List

from fastapi import FastAPI

from app.schemas.models import AssistantResponse, Product, UserRequest
from app.services.constraint_solver import filter_products
from app.services.model_client import ServiceUnavailable, get_intent

app = FastAPI()
CATALOG: List[Product] = []


@app.post("/assist", response_model=AssistantResponse)
def assist(request: UserRequest) -> AssistantResponse:
    try:
        intent = get_intent(request.text)
    except ServiceUnavailable:
        return AssistantResponse(
            intent={"type": "clarify"},
            clarifying_question=(
                "I cannot interpret that request right now. "
                "Would you like to restate the product and constraints?"
            ),
            notes=["Intent service is temporarily unavailable."],
        )

    if intent.type == "clarify":
        return AssistantResponse(
            intent=intent,
            clarifying_question=(
                "What product are you looking for, and which constraints matter?"
            ),
        )

    if intent.type == "bundle_request":
        return AssistantResponse(
            intent=intent,
            clarifying_question=(
                "Bundle requests are handled by the bundle workflow. "
                "Which required items and total budget should it use?"
            ),
        )

    if intent.type == "checkout_intent":
        return AssistantResponse(
            intent=intent,
            clarifying_question=(
                "Checkout requires the separate authenticated checkout workflow."
            ),
        )

    if not intent.categories:
        return AssistantResponse(
            intent=intent,
            clarifying_question="Which product category would you like to search?",
        )

    recommendations = filter_products(CATALOG, intent)
    notes = [] if recommendations else [
        "No catalog item satisfied every supplied constraint."
    ]

    return AssistantResponse(
        intent=intent,
        recommendations=recommendations,
        notes=notes,
    )

The endpoint does not convert a bundle request or checkout phrase into a transaction. Those intents are routed toward separate workflows instead of receiving an unrelated category question. Reservation, payment, and order creation should remain authenticated operations with authorization and idempotency controls.

3.7 Retries and a Compact Circuit Breaker

Add the openai package to requirements.txt, and choose mutually compatible current releases of FastAPI, Pydantic, and the other packages instead of installing the earlier illustrative dependency list unchanged.

This example sets max_retries=0 on the OpenAI client so Tenacity owns the retry policy instead of stacking two retry layers. It retries only the transport and service exceptions exported by the intent module. A refusal, incomplete parse, or semantic validation failure becomes a clarify result and is not retried.

# app/utils/backoff.py
from tenacity import retry, retry_if_exception_type
from tenacity import stop_after_attempt, wait_exponential

from app.llm.intent import RETRYABLE_ERRORS

retry_model_call = retry(
    retry=retry_if_exception_type(RETRYABLE_ERRORS),
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=0.5, min=0.5, max=4),
    reraise=True,
)

The circuit breaker stops incoming requests from repeatedly calling a dependency after consecutive failures. A lock protects its state because synchronous FastAPI handlers can execute concurrently.

# app/utils/circuit_breaker.py
import threading
import time


class CircuitOpen(RuntimeError):
    pass


class CircuitBreaker:
    def __init__(self, failure_threshold: int, cooldown_seconds: float):
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.failures = 0
        self.opened_at = 0.0
        self._lock = threading.Lock()

    def before_call(self) -> None:
        with self._lock:
            if self.failures < self.failure_threshold:
                return
            elapsed = time.monotonic() - self.opened_at
            if elapsed < self.cooldown_seconds:
                raise CircuitOpen("intent circuit is open")
            self.failures = 0
            self.opened_at = 0.0

    def record_success(self) -> None:
        with self._lock:
            self.failures = 0
            self.opened_at = 0.0

    def record_failure(self) -> None:
        with self._lock:
            self.failures += 1
            if self.failures >= self.failure_threshold:
                self.opened_at = time.monotonic()

# app/services/model_client.py
from app.llm.intent import extract_intent
from app.schemas.models import ParsedIntent
from app.utils.backoff import retry_model_call
from app.utils.circuit_breaker import CircuitBreaker, CircuitOpen

breaker = CircuitBreaker(
    failure_threshold=3,
    cooldown_seconds=30.0,
)


class ServiceUnavailable(RuntimeError):
    pass


@retry_model_call
def _retrying_extract(text: str) -> ParsedIntent:
    return extract_intent(text)


def get_intent(text: str) -> ParsedIntent:
    try:
        breaker.before_call()
    except CircuitOpen as exc:
        raise ServiceUnavailable("intent circuit is open") from exc

    try:
        intent = _retrying_extract(text)
    except Exception as exc:
        breaker.record_failure()
        raise ServiceUnavailable("intent extraction failed") from exc

    breaker.record_success()
    return intent

Together, app/llm/intent.py and app/services/model_client.py implement the model-wrapper responsibility shown in the project layout. The first owns schema parsing; the second owns retry and availability behavior. The separate app/utils/backoff.py and app/utils/circuit_breaker.py modules keep those policies testable.

This breaker coordinates threads inside one process. Deployments with multiple worker processes need shared breaker state or must accept that each worker has an independent circuit. The /assist handler converts an open circuit or exhausted retry sequence into a controlled degraded response.

The k8s/ files in the project layout follow the usual pattern for a stateless FastAPI service, with credentials and configuration supplied outside the container.

Section 4: Testing and Operating the Assistant

Test constraint behavior without making a model call. This keeps failures local and makes each catalog rule easy to reproduce.

# tests/test_constraint_solver.py
from app.schemas.models import ParsedIntent, Product
from app.services.constraint_solver import filter_products


def test_filter_rejects_products_that_break_hard_constraints():
    products = [
        Product(
            id="eligible",
            title="Plain Cotton Shirt",
            category="shirts",
            price=60,
            sizes=["M"],
            materials=["cotton"],
            in_stock=True,
            shipping_days=1,
        ),
        Product(
            id="wrong-category",
            title="Cotton Walking Shoe",
            category="shoes",
            price=55,
            sizes=["M"],
            materials=["cotton"],
            in_stock=True,
            shipping_days=1,
        ),
        Product(
            id="too-slow",
            title="Plain Cotton Shirt",
            category="shirts",
            price=55,
            sizes=["M"],
            materials=["cotton"],
            in_stock=True,
            shipping_days=4,
        ),
        Product(
            id="excluded",
            title="Printed Cotton Shirt",
            category="shirts",
            price=50,
            sizes=["M"],
            materials=["cotton"],
            in_stock=True,
            shipping_days=1,
        ),
    ]
    intent = ParsedIntent(
        categories=["shirts"],
        size="M",
        price_max=70,
        materials=["cotton"],
        exclude=["printed"],
        shipping_speed="next-day",
    )

    results = filter_products(products, intent)

    assert [item.product_id for item in results] == ["eligible"]

Add boundary cases for an exact maximum price, empty material lists, out-of-stock products, and conflicting minimum and maximum prices. Test the model client separately with mocked SDK responses for successful parsing, refusal, incomplete output, invalid values, exhausted retries, and an open circuit.

  • Protect customer data: Log normalized intent fields and internal correlation identifiers, not raw personal data or complete conversation text.
  • Maintain an evaluation set: Keep representative shopping queries with reviewed expected intents and constraint outcomes. Run them when prompts, schemas, models, or normalization rules change.
  • Measure fallback behavior: Track refusal, incomplete-response, validation-failure, clarification, retry, and open-circuit rates as separate signals.
  • Keep catalog facts current: Refresh price, inventory, variant, and delivery data from authoritative systems, then recheck transaction-critical values before reservation or checkout.
  • Inspect recommendation quality: Review whether eligible items are omitted, excluded items appear, or ranking explanations disagree with the score calculation.
  • Limit stored context: Retain only session fields needed for the next turn, follow the organization’s retention rules, and keep secrets and payment data out of prompts.

Conclusion

A dependable shopping assistant separates language interpretation from catalog truth. The model converts a request into typed intent, while deterministic code validates constraints, filters products, and blocks unsupported actions.

The completed service includes schema-based parsing, refusal and incomplete-response handling, hard category and product filters, transparent ranking, a FastAPI endpoint, targeted retries, a thread-safe circuit breaker, a constraint-filter test, and guidance for testing the model client. Connect these boundaries to current catalog and fulfillment services, then use evaluation results and operational signals to refine prompts and rules without weakening transaction controls.

Get Free Access to 40,000+ AI Prompts for ChatGPT, Claude & Codex

Subscribe for instant access to the largest curated Notion Prompt Library for AI workflows.

Markos Symeonides

More on this