How to Cut OpenAI and LLM Token Costs Without Cutting Product Quality

Executive summary

The expensive default in an LLM product is rarely one badly worded prompt. It is the accumulation of unnecessary requests, oversized context, excessive reasoning, verbose output, repeated cache misses, and a frontier model applied to work that never needed it. Once those choices disappear into infrastructure, the product can spend heavily while every individual request still looks harmless.^{1, 2, 3, 4}

I use CodexToken in this article as a practical shorthand for token consumption across OpenAI- and LLM-powered applications. It is not an OpenAI billing unit or a separate product. The useful idea behind the term is that token efficiency belongs in system architecture, product policy, and operations—not in a collection of prompt tricks.¹

The order I would use is: requests → model → input context → reasoning → output → cache → serving tier. First avoid generations that do not need to exist. Then route the task to the least expensive model that meets a measured quality threshold. Only after that should the team tune context, reasoning, output, caching, and asynchronous processing.^{1, 6, 7}

At the 13 August 2026 US Standard API reference, GPT-5.6 Sol costs $5 per million uncached input tokens, $0.50 per million cached input tokens, and $30 per million output tokens. Terra costs $2, $0.20, and $12; Luna costs $0.20, $0.02, and $1.20. GPT-5.6 cache writes cost 1.25 times the uncached-input rate. These are dated operating inputs, not constants, so production pricing should live in configuration and be rechecked before a forecast or release.^{1, 2, 3, 4}

All three models currently expose a 1,050,000-token context window, with up to 922,000 input tokens and 128,000 output tokens. That capacity should not be mistaken for a target. Above 272,000 input tokens, the full GPT-5.6 request moves to long-context pricing: twice the input rate and one-and-a-half times the output rate. A large window removes a hard constraint; it does not remove the budget.^{2, 3, 4}

Why token economics becomes a product problem

A prototype can hide token waste for months. A product cannot. Consider a support copilot that receives 100,000 requests. If each request sends 10,000 input tokens and generates 1,000 output tokens on GPT-5.6 Terra, the Standard API arithmetic is $0.032 per request, or $3,200 per 100,000 requests. The number is easy to calculate; the harder question is where those tokens came from.^{1, 3}

The user may have typed “Can I get a refund?” while the application silently added a 5,000-token system policy, 2,000 tokens of tool schemas, 1,500 tokens of conversation history, 1,000 tokens of retrieved documentation, and 500 tokens of current task data. The application contributed almost the entire bill. This is why a product team should inspect the request assembled at the API boundary, not judge efficiency from the text box.^{9, 8}

OpenAI's input-token counting endpoint accepts the same request shape as a Responses API call. It can count messages, tools, schemas, files, images, and conversation structure that a character-count heuristic misses. Count before inference and treat the result as a routing input: compact the request, retrieve less, reject it, or move it to an explicitly approved high-context route.⁹

Seven forces usually explain the drift: context inflation, verbose output, reasoning effort, an overpowered model, retries and regeneration, tool or retrieval overhead, and long-context convenience. previous_response_id can simplify state management, but earlier input in the chain is still billed as input. A clean application state model is useful; it is not a cost waiver.^{8, 9, 3}

Output deserves separate attention because current GPT-5.6 output tokens cost six times uncached input tokens on Sol and Terra. Reasoning tokens are also billed as output and occupy the generated-token budget. A request can therefore look compact at the input boundary and still become expensive after an unbounded reasoning or verbosity setting.^{1, 2, 3}

Tokens per request are useful for engineering, but cost per successful task is the more durable business measure. Uncached input, cached reads, cache writes, reasoning, output, retries, and model escalation have different prices and different effects on whether the task succeeds. A cheaper request that triggers another request is not necessarily a cheaper task.^{1, 5}

The token-efficiency stack

There is no universal “70% token saving” switch. A useful stack combines tactics that do three different jobs: remove requests or tokens, lower the price of tokens that remain, and reduce perceived waiting without pretending the bill changed. Keeping those mechanisms separate prevents teams from celebrating a faster spinner as a cost reduction.^{1, 5, 6, 7}

Model selection should precede prompt micro-optimization. For the same 10,000-input and 1,000-output workload, current Standard pricing yields about $0.08 on Sol, $0.032 on Terra, and $0.0032 on Luna before cache effects. Luna is roughly 25 times cheaper than Sol for that token mix because the tokens are priced differently, not because fewer tokens were used. The route is valid only if representative evaluations show that Luna completes the work.^{1, 2, 3, 4}

A budget-aware router prepares and classifies the request before choosing a model. It removes deterministic clutter, retrieves and deduplicates external evidence, counts the assembled input, scores task difficulty and risk, selects a focused, normal, or complex route, and attaches separate reasoning and output ceilings. The response then feeds cost, latency, cache, retry, and quality evidence back into the router.^{9, 2, 3, 4}

Reasoning needs its own budget. GPT-5.6 supports effort levels from none to max, but maximum effort should be an explicit route rather than a default. max_output_tokens limits the total generated budget, including reasoning and visible output, so the product must reserve enough room for the answer after setting the reasoning policy.^{2, 3, 4}

Temperature is not a cost limit. It changes sampling behavior, not the maximum amount generated. Likewise, automatic truncation is a safety valve rather than a context strategy. Remove irrelevant HTML, duplicate text, repeated policies, obsolete conversation turns, and unused tool schemas deliberately; then count the complete request again.^{9, 8}

Prompt and code patterns that save tokens

Prompt optimization works when repeated prose becomes a compact contract. The goal is not telegraphic writing. It is to remove semantic redundancy while preserving the task, input, output schema, and the rules that actually change behavior.^{9, 5}

The pattern is straightforward: TASK, INPUT, OUTPUT, and only the RULES that change the result. Keep the stable portion versioned; put changing user data after it. This makes the prompt easier to review and creates an exact reusable prefix for eligible caching instead of invalidating the cache with timestamps, random request IDs, or user-specific text at the top.⁵

OpenAI prompt caching applies to eligible prompts of at least 1,024 tokens and depends on exact shared prefixes. GPT-5.6 supports explicit prompt_cache_breakpoint and prompt_cache_key controls. A cache read is cheaper, but the first write is more expensive than ordinary uncached input, so report writes and reads separately and calculate how many future hits each stable prefix actually receives.^{5, 1}

Zero-shot first is a sensible operating default; examples should earn their place. A few-shot example can increase accuracy enough to avoid retries or permit a cheaper model, in which case it may reduce cost per successful task even while adding input tokens. The decision belongs to an evaluation, not to a universal rule about prompt length.^{9, 3}

Dynamic context beats infinite context. Instead of attaching an 80,000-token manual to every question, filter by metadata, retrieve with semantic and lexical signals, deduplicate, rerank, and send the small set of evidence that answers the current request. For long conversations, preserve current goals, decisions, unresolved work, and durable user facts while compacting low-value turn-by-turn history.^{9, 8}

A practical gateway can be expressed without a large framework: look up the route, count the assembled request with client.responses.input_tokens.count, reject or compact input above the route ceiling, call client.responses.create with the selected model and max_output_tokens, then record usage and task outcome. The crucial behavior is not the Python syntax. It is that a request over budget cannot silently proceed.^{9, 2, 3, 4}

Product, policy, and operations matter as much as prompts

The largest saving can happen before the prompt exists. Progressive disclosure lets a product return three issues and three next actions before it generates a full report. Preview-first writing can offer an outline, one section, or the full document with different budgets. A conversational “thanks” can receive a deterministic response without replaying a long session to a reasoning model.^{9, 8}

On-device or server-side preprocessing can strip navigation, normalize whitespace, remove duplicate blocks, redact unnecessary identifiers, extract structured fields, and detect language or intent before a model sees the input. This can reduce both tokens and exposure of irrelevant data. It must still preserve information that the task and the user's consent require.^{9, 8}

Streaming should be sold internally as a latency and interaction feature. Presenting output early makes an application feel faster, and intentional cancellation can stop generation once the user has enough. Simply streaming the same 1,000-token answer does not turn it into a 500-token answer.¹

Model access should be governed like infrastructure permission. A focused route can handle classification, extraction, normalization, and other narrow work. A balanced route can handle ordinary generative tasks. A frontier route can be reserved for difficult, high-risk, or high-value work that shows a measurable gain. The exact assignment is product-specific; the escalation evidence should not be optional.^{2, 3, 4}

Batch everything that does not need an immediate answer. OpenAI's Batch API currently costs 50% less than synchronous processing, has a separate rate-limit pool, and completes within a 24-hour window. Flex uses Batch token rates for supported models in exchange for slower responses and possible temporary resource unavailability. Embedding backfills, nightly classification, analytics enrichment, evaluations, and bulk summarization are natural candidates.^{6, 7}

Quotas must also exist above the API. Limit autonomous turns, retries, tool calls, model escalations, reasoning effort, and tenant spend. Attribute cost to a feature, user, tenant, route, and successful task so a budget owner can distinguish legitimate high-value work from a loop or a badly configured default.^{1, 8}

Return to the Terra example. If engineering reduces the request to 500 uncached input tokens, 1,500 reusable cached input tokens, and 300 output tokens, a steady-state cache-hit illustration costs about $0.0049 per request at the checked rates: $0.001 uncached input, $0.0003 cached input, and $0.0036 output. That is about 84.7% lower than $0.032, or about $490 rather than $3,200 per 100,000 requests.^{1, 3, 5}

That comparison is intentionally simplified. It assumes the prefix is already cached and every illustrated request is a hit. Real accounting must include cache misses and the 1.25-times cache-write price, then amortize those writes across the actual number of reads. The saving comes from context reduction, prefix reuse, and a smaller output—not one clever prompt.^{1, 5}

Risks, KPIs, and the team checklist

Token optimization becomes dangerous when “remove waste” turns into “make the number as small as possible.” A cheaper assistant that answers the wrong question is not optimized. It is broken. Every cost change should be paired with a representative offline evaluation, an online quality signal, and a rollback boundary.^{2, 3, 4}

The main risks are predictable: quality can fall when examples or evidence disappear; latency can rise when Batch or Flex is applied to an interactive path; quotas can create UX friction; compaction can erase state; and telemetry can create a privacy problem if a team logs raw prompts instead of the minimum usage and attribution data it needs. Responses are stored for 30 days by default unless storage is disabled, while conversation objects have different persistence semantics, so data lifecycle and cost telemetry should be designed together.^{6, 7, 8}

For implementation, begin with a feature inventory. Give summarization, extraction, chat, coding, and deep research different input, reasoning, and output ceilings. Count the complete request. Evaluate the cheapest model that meets the threshold. Retrieve and compact instead of sending the world. Put stable prefixes before dynamic content. Move asynchronous work to Batch or Flex. Bound loops and retries. Keep task success beside cost.^{9, 5, 6, 7}

The operating loop is simple: observe production usage; attribute it to the route and task; inspect a budget or quality regression; change context, model, reasoning, output, cache, or serving tier; run offline and online evaluations; roll out gradually; and return the evidence to the router. This is cloud cost engineering for model context.^{1, 9}

The mature application does not say, “Here is everything we know; think as hard as possible; return a comprehensive answer.” It says, “Here is what this task needs; use the least expensive route that can solve it; reason only as much as necessary; return the useful result; reuse what we already paid to process.” That is what CodexToken saving means in practice: every expensive token has a reason to exist.^{1, 5, 3}