Batch API vs Standard API: The Real Cost After Reruns

Publication note: the underlying price rows and this article were separately approved by the site owner on 2026-08-23. Provider rules and rates were checked on 2026-08-22. All workloads and dollar results are simulated, not customer bills or production benchmarks.

Batch APIs advertise a compelling trade: accept asynchronous processing and pay roughly half the normal token price. That discount is real for the supported OpenAI, Anthropic, and Gemini examples in this experiment, but “50% lower token rates” is not the same statement as “your application will cost 50% less.” A batch pipeline has job construction, identifiers, polling or webhooks, result reconciliation, quality checks, retries, deadlines, and data-retention decisions. It may also be the wrong interface when request B depends on the immediate output of request A.

This experiment asks two practical questions. First, what do three representative offline workloads cost at Standard and Batch token rates? Second, how much of the headline discount remains when a small portion of completed, billed results fails an application-level quality gate and must be submitted again?

The evidence is available as batch-cost-scenarios.json and batch-cost-scenarios.csv. A build-time verification script recalculates every row from the approved TokenCostAI catalog and the official Batch discounts. The comparison covers text input and output tokens only. It excludes tool-call fees, context-cache writes or storage, long-context modifiers, taxes, file storage, networking, engineering labor, and the economic cost of waiting.

What “Batch” means in this comparison

Standard APIs are used here as the synchronous, full-token-price baseline. An application sends a request and expects the response as part of the current workflow. Batch APIs collect independent requests into an asynchronous job. The application submits the job, waits, checks its status, downloads or streams results, then maps each result back to the original input.

That architecture fits nightly summaries, large evaluation suites, document enrichment, embeddings, and other offline work. It does not fit a chat reply that a person is waiting to read, a checkout decision that must complete now, or an agent loop where the next call cannot be constructed until the previous call returns. Gemini separately documents Flex inference for some non-urgent sequential cases, but Flex is not Batch and is outside this experiment.

The providers also differ in details that affect engineering work:

Provider	Confirmed discount	Timing	Important operational limit
OpenAI	50% vs synchronous token rates	24-hour completion window	Up to 50,000 requests and 200 MB per batch file
Anthropic	50% vs standard API prices	Most finish under one hour; expire at 24 hours	Up to 100,000 requests or 256 MB, whichever comes first
Gemini	50% vs interactive API	Target turnaround of 24 hours	Inline jobs under 20 MB; file input up to 2 GB
DeepSeek	not confirmed	not comparable	No direct Batch endpoint or Batch rate located in reviewed official docs

“Not confirmed” is deliberate. It does not claim that DeepSeek can never introduce or expose batch processing. It means this article does not have primary-source evidence for a direct DeepSeek Batch price on the review date, so it will not manufacture a 50% rate.

The cost formula, including accepted-result reruns

For each workload, total input and output tokens are converted to millions. Standard cost is:

input MTok × standard input rate + output MTok × standard output rate

The first Batch attempt uses the corresponding Batch input and output rates. The simulation then applies an accepted-result rerun rate:

Batch with reruns = first Batch cost × (1 + rerun rate)

The phrase “accepted-result rerun” matters. It means a request completed and was billed, but the application rejected the result because it failed a schema check, contained missing fields, exceeded a quality threshold, or otherwise needed to be generated again. It is not an assumption that every provider bills validation errors, expired requests, or provider-side failures in the same way. Actual API errors must be classified from result files and reconciled against billing data.

When both Batch input and output rates are exactly half of Standard, a 5% accepted-result rerun rate leaves a 47.5% token saving, a 10% rerun rate leaves 45%, and a 2% rerun rate leaves 49%. This simple relationship is useful, but it should not hide absolute cost or operational risk.

Scenario 1: nightly support summaries

The first simulated workload contains 100,000 independent support conversations. Each request uses 2,000 input tokens and produces 300 output tokens, for a total of 200 million input tokens and 30 million output tokens. Results are collected overnight, so an immediate response is unnecessary. The quality gate causes 5% of completed results to be rerun.

Model	Standard	First Batch	Batch + 5% reruns	Saving
GPT-5.6 Sol	$1,900.00	$950.00	$997.50	$902.50
Claude Haiku 4.5	$350.00	$175.00	$183.75	$166.25
Gemini 3.7 Flash	$262.50	$131.25	$137.8125	$124.6875

These rows are not a model-quality ranking. Each model may produce different acceptance rates, token counts, or task outcomes. The table holds the workload and rerun percentage constant only to demonstrate the billing formula. A real deployment must evaluate whether each model can perform the task and measure its own output length and acceptance rate.

The workload also requires partitioning decisions. OpenAI’s 50,000-request maximum means at least two batches for 100,000 requests. Anthropic allows up to 100,000 requests if the 256 MB size limit is not reached. Gemini may use file input for large jobs. Those job counts do not change token math, but they change orchestration, monitoring, and recovery.

Scenario 2: offline model evaluation

The second workload runs 50,000 independent evaluation cases with 5,000 input tokens and 1,000 output tokens per case. The totals are 250 million input tokens and 50 million output tokens. Evaluation work is well suited to asynchronous execution because the aggregate report can wait; however, structured-output or scoring failures may require reruns. This simulation uses a 10% accepted-result rerun rate.

Model	Standard	First Batch	Batch + 10% reruns	Saving
GPT-5.6 Sol	$2,750.00	$1,375.00	$1,512.50	$1,237.50
Claude Haiku 4.5	$500.00	$250.00	$275.00	$225.00
Gemini 3.7 Flash	$375.00	$187.50	$206.25	$168.75

Batch does not make the evaluation reproducible by itself. Save the dataset version, model identifier, relevant parameters, prompt template, request custom_id or equivalent key, and the result status. Output order may differ from input order, so array position is not a safe join key. OpenAI and Anthropic explicitly tell developers to reconcile results through custom identifiers.

This is also where selective retry matters. If 5,000 of 50,000 results fail the application quality gate, create a new job containing those 5,000 identifiers. Re-submitting the entire original job would duplicate successful work. Gemini’s documentation explicitly states that job creation is not idempotent: submitting the same creation call twice produces two separate jobs. A timeout in the client must therefore trigger a status lookup or idempotency mechanism in the application, not an automatic blind create retry.

Scenario 3: bulk record extraction

The third workload extracts fields from 200,000 records. Each request uses 1,200 input tokens and produces 100 output tokens, totaling 240 million input and 20 million output tokens. The output is compact and schema-oriented, and the simulation assumes only 2% of completed results need another attempt.

Model	Standard	First Batch	Batch + 2% reruns	Saving
GPT-5.6 Sol	$1,800.00	$900.00	$918.00	$882.00
Claude Haiku 4.5	$340.00	$170.00	$173.40	$166.60
Gemini 3.7 Flash	$255.00	$127.50	$130.05	$124.95

Two hundred thousand records exceed the per-batch request maximum for both OpenAI and Anthropic, so the dataset must be split even if the encoded file would fit. Stable identifiers must survive that split. A practical key can combine the dataset version and record ID, such as customers-2026-08-record-004213, while staying within the provider’s identifier rules.

Low output volume makes input pricing dominate this example. If records share a large stable prefix, prompt or context caching might lower input cost further, but caching rules interact differently with each Batch service. Mixing caching into this baseline would obscure the Batch-only result. Test and report that as a separate billing category rather than silently assuming every repeated prefix hits.

OpenAI: separate capacity, fixed window, partial expiration

OpenAI documents a 50% discount compared with synchronous APIs, a separate pool of Batch rate limits, and a 24-hour turnaround. A single batch may include up to 50,000 requests with an input file up to 200 MB. Output order may differ from input order, so custom_id is required for reliable reconciliation.

Expiration is partial, not necessarily all-or-nothing. If a job reaches the deadline, unfinished requests are cancelled, completed responses remain available, and completed requests are billed. An error file identifies expired requests. A retry worker should therefore submit only expired or otherwise retryable identifiers. It should not assume that the whole batch was free or failed.

Official references: OpenAI Batch guide and GPT-5.6 Sol model page.

Anthropic: independent messages and finite result retention

Anthropic says all active Claude models support Message Batches and all usage is charged at 50% of standard API prices. A batch is limited to 100,000 requests or 256 MB. Most batches finish in under one hour, but requests can expire at 24 hours. One request failing does not fail the rest of the batch.

Results are available for 29 days after creation. That is a retrieval window, not an archival strategy. Download, verify, and store the results according to the application’s own retention policy before the provider window closes. The platform also notes that batch processing stores request and response data during asynchronous processing, so teams with data-governance requirements must review retention and workspace isolation rather than evaluating price alone.

The experiment uses Claude Haiku 4.5 because its approved standard $1 input and $5 output rates and official $0.50/$2.50 Batch rates are stable in the reviewed evidence. It avoids silently applying a soon-changing scheduled rate from another model.

Official references: Anthropic Batch processing and Claude pricing.

Gemini: large files, per-line errors, and non-idempotent creation

Gemini’s Batch API is priced at 50% of the equivalent interactive API for supported models and targets a 24-hour turnaround. Small jobs can use inline requests under 20 MB; large jobs can use JSONL file input, with a documented 2 GB file limit. The experiment uses Gemini 3.7 Flash’s promotional rates through 2026-12-31: $0.75/$3.75 Standard input/output and $0.375/$1.875 Batch input/output. Recalculate after the promotional period.

Gemini instructs developers to inspect failedRequestCount and parse each output line for either a response or an error status. Creating a job is not idempotent. These two facts should shape the retry design: persist the created job name, reconcile every record, and create a follow-up job only from selected failures. A network timeout after creation must not be treated as proof that creation failed.

Context caching is supported in batch requests, but a cache hit is billed under the documented context-caching rules. This article does not replace those rates with the Batch input price. HV-02 covers cache economics separately.

Official references: Gemini Batch API and Gemini API pricing.

DeepSeek: do not invent a missing Batch rate

The official DeepSeek documentation reviewed on 2026-08-22 provides direct API pricing and automatic context-cache behavior, but this research did not locate a direct Batch endpoint or Batch token rate. DeepSeek is therefore not comparable in the Batch tables. Its Standard rate is retained only to identify the reviewed model; no half-price row is calculated.

This is the correct treatment for missing evidence. An OpenAI-compatible request shape, a third-party queue, or an application that sends many normal requests asynchronously is not automatically the provider’s discounted Batch product. If DeepSeek publishes an official endpoint and rate later, add it through the same price review and article approval gate.

Official documentation entry points: DeepSeek API docs and DeepSeek pricing.

A failure-safe Batch workflow

First, validate one representative request synchronously. This catches invalid model names, unsupported parameters, schema problems, and prompt defects before thousands of copies enter a job. Next, assign a stable unique identifier to every request and write an immutable manifest containing the dataset version, prompt version, model, expected row count, and token estimate.

Split the manifest according to both request-count and file-size limits. Submit each job once, then store the returned job identifier before starting another. Poll conservatively or use a supported webhook, and treat an uncertain create response as a reconciliation problem rather than permission to create a duplicate.

When a job reaches a terminal state, download results and errors, match by identifier, and account for every input row. Separate successful, validation-error, provider-error, expired, cancelled, and application-rejected results. Only the appropriate categories should enter a retry manifest. Record the retry generation so a result cannot be imported twice.

Finally, calculate actual cost from provider usage and billing exports. Compare accepted results per dollar, not just submitted requests per dollar. A nominal 50% token discount can coexist with wasted duplicate jobs, excessive output, poor acceptance rates, or labor-intensive recovery.

When Standard remains the better choice

Use Standard APIs when a human or dependent workflow needs the answer now, when requests are inherently sequential, when the dataset is too small to justify orchestration, or when the provider/model combination lacks confirmed Batch support. Standard can also be safer during prompt development because errors are visible immediately.

Batch becomes attractive after the request shape is stable, work is independent, the completion window is acceptable, and identifiers and reconciliation are implemented. The discount compensates the application for surrendering immediacy and managing asynchronous state; it does not remove the responsibility to test quality or recover partial results.

What these results do not prove

These results are simulated from official public rules and rates checked on 2026-08-22. They do not measure output quality, latency distributions, reliability, queue congestion, total cloud cost, or engineering effort. The fixed rerun percentages are scenario inputs, not provider failure rates. The data is not a real-time feed.

The model rows are not a capability ranking and should not be used to choose a provider solely by the lowest dollar total. Re-run the included formulas with current rates, actual token usage, observed acceptance rates, and the specific provider’s billing export before making a purchasing or migration decision.