How to Cut OpenAI API Costs by 80% — Real Numbers and Tactics

A client came to me with a $1,200/month OpenAI bill that was still climbing. They were calling 4o on every request with an 8,000-token prompt, 5,000 calls a day. Two weeks of optimisation later, the same workload was costing $240. These are the eight specific tactics that worked.

Tactic 1: Right-size the model

GPT-4o is $2.50/$10 (per 1M input/output tokens). GPT-4o-mini is $0.15/$0.60 — sixteen times cheaper. Most teams default to 4o for everything when mini would handle the job just as well. For every project I run an eval suite of 50 real requests, test both, and downshift to mini wherever it holds up.

Tactic 2: Turn on prompt caching

OpenAI added prompt caching in 2024. If you reuse the same prompt prefix (>1024 tokens) within 5 minutes, the cached portion is billed at 50% off. System prompts, RAG context, instructions — these don't change. Put them at the start, put the dynamic user input at the end.

# YANLIŞ — dynamic məlumat ortadadır, cache işləmir
messages = [
    {"role": "system", "content": INSTRUCTIONS},
    {"role": "user", "content": f"Kontekst: {user_specific_data}\n\n{LARGE_RAG_CONTEXT}\n\nSual: {question}"}
]

# DÜZGÜN — static hissə əvvəldə, dynamic sona doğru
messages = [
    {"role": "system", "content": INSTRUCTIONS + "\n\n" + LARGE_RAG_CONTEXT},
    {"role": "user", "content": f"{user_specific_data}\n\nSual: {question}"}
]

Tactic 3: Batch API — 50% off

If you don't need real-time responses (overnight reports, content processing, embedding generation), use the Batch API. Completes within 24 hours, costs half as much. One client was embedding 2,000 documents nightly — synchronous API cost $80/month, batch dropped it to $40.

Tactic 4: Cap max_tokens

By default OpenAI lets the response run up to the model's context window. So your bot might write a 4,000-token reply when you needed 200. Set max_tokens=200 — costs drop and your answers get more concise as a bonus.

Tactic 5: Trim the context you don't need

I was building a chat bot that sent the full conversation history every turn. By message 20 the prompt was 5,000 tokens. Switched to keeping only the last 6 messages, with everything older compressed into a short summary. Token usage dropped 70%, response quality basically unchanged.

def build_context(history: list, max_recent=6):
    if len(history) <= max_recent:
        return history

    old = history[:-max_recent]
    recent = history[-max_recent:]

    summary = summarize(old)  # 3 cümləlik xülasə
    return [{"role": "system", "content": f"Əvvəlki söhbət xülasəsi: {summary}"}] + recent

Tactic 6: Cache your embeddings

In a RAG system you don't need to embed the same document five times. I cache every embedding in Postgres with pgvector, deduped by content hash. A client was re-embedding the same 1,000 product descriptions every day — turned out to be $30/day. After caching: $2/day.

Tactic 7: Validate with structured output

If responses don't match the format you expect, you retry — and pay twice. Use response_format={'type': 'json_schema', ...} to force valid output. One client's retry rate went from 22% to 0%, an effective 22% cost reduction with no other changes.

Tactic 8: Stream for monitoring

Not strictly a cost tactic but a debugging multiplier. With streaming on, I log every chunk and timing. Makes it easy to spot slow requests and fix them. One endpoint was painfully slow — log showed the first token took 8 seconds. Swapped 4o for 4o-mini, dropped to 1.2s.

Final numbers

• Starting: $1,200/month
• Switch to mini: $470 (-61%)
• Prompt caching: $385 (-18%)
• Context trimming: $295 (-23%)
• Embedding caching: $245 (-17%)
• Final: $240/month — 80% reduction

If your OpenAI bill is higher than you'd like, I'm happy to audit it — usually a 30-minute call is enough to surface 2-3 concrete savings.