Cheapest LLM APIs: compare total workload cost, not token price alone

A search for the cheapest LLM API usually ends at a table of input and output prices per million tokens. That table answers one question: what a single successful call costs on paper. Your bill, and your agent’s uptime, depend on what happens around that call: how many tokens the model writes, how often it gets throttled, and how often you throw the answer away and ask again.

This guide gives you a way to compare a cheap LLM API against a free model API on the work you actually run, using prices published by Groq and Google as worked examples.

Start with the published token prices

Token prices are the right starting point, as long as you read them carefully.

Groq’s supported models page lists, for production models, a price per million input and output tokens next to speed and context window. On 7 November 2026 it showed openai/gpt-oss-20b at $0.075 input and $0.30 output, and openai/gpt-oss-120b at $0.15 input and $0.60 output, both with a 131,072-token context window. The same table lists llama-3.3-70b-versatile and llama-3.1-8b-instant as “ContactSales” rather than with a public price. Groq also separates the two model classes: “Preview models are intended for evaluation purposes only and should not be used in production environments.”

Google’s Gemini API pricing page splits every model into a free tier and a paid tier. On the same date, gemini-3.5-flash-lite cost $0.30 per million input tokens and $2.50 per million output tokens on the paid Standard tier, and was marked free of charge on the free tier. gemini-3.8-flash was $0.75 input and $3.75 output through 31 December 2026, rising to $1.50 and $7.50 from 1 January 2027.

Two details on those pages already change the ranking:

• Prices carry dates. Gemini 3.8 Flash doubles on 1 January 2027. A comparison you made in November is wrong in January.
• Output is billed differently. Google labels the column “Output price (including thinking tokens)”. A model that reasons before answering bills that reasoning as output, at the higher rate.

The workload-cost formula

To compare two low-cost LLM APIs fairly, estimate the cost of one accepted result rather than one call:

cost per accepted result = (input cost + output cost) × attempts per task ÷ share of results you keep

Each term maps to something you can measure on your own traffic:

1. Input cost. Prompt tokens, including system prompt, tool definitions, and conversation history, times the input price.
2. Output cost. Visible answer tokens plus any thinking tokens, times the output price.
3. Attempts per task. Retries after timeouts, server errors, or throttling that eventually succeed.
4. Share of results you keep. Answers that pass your check: valid JSON, a correct label, a test that passes, or a human reviewer’s approval.

A worked example with published prices

Take a request with 2,000 input tokens and 500 output tokens. At the prices above:

Model (paid tier)	Input cost	Output cost	Per request	Per 10,000 requests
Groq openai/gpt-oss-20b	$0.00015	$0.00015	$0.00030	$3.00
Gemini gemini-3.5-flash-lite	$0.00060	$0.00125	$0.00185	$18.50

On token price alone, the Groq model is about six times cheaper for this request shape. Now change one input at a time. These are hypothetical values to show how the formula moves, not measurements of either model:

• If the cheaper model needs a second attempt on half its tasks (1.5 attempts per task) and you keep only 60% of its answers, its cost per accepted result rises to $0.00075, or $7.50 per 10,000 accepted results.
• If the other model’s answer is longer because it reasons first, every extra output token costs $2.50 per million, more than eight times its input rate.

Your measured attempt rate and keep rate decide the winner. Neither price table contains them.

Costs that never appear in a price table

Rate limits turn into latency and retries

Groq’s rate limits documentation states that limits “apply at the organization level, not individual users,” and that exceeding one returns HTTP 429. The retry-after header gives the wait in seconds, and headers such as x-ratelimit-remaining-tokens and x-ratelimit-reset-requests show how close you are to the next limit. A 429 costs you the time spent waiting and the code that decides whether to wait, retry, or move on.

Google’s Gemini rate limits page applies limits “per project, not per API key”, resets daily quotas at midnight Pacific time, and warns that “specified rate limits are not guaranteed and actual capacity may vary.” Creating a second key in the same project adds no capacity.

For an agent loop, a model that is cheap per token but throttled every few minutes can cost more in stalled work than a pricier one that answers.

Free tiers have a price that is not money

A free model API removes the token line from the formula but adds conditions. Google’s pricing page says free-tier content is “used to improve our products”, while paid-tier content is not. It also gives the free tier “limited access to certain models.” Groq’s provider page and Gemini’s provider page record the current free-plan scope, quota units, and data terms for each.

Free access is also uneven between providers. A zero-price catalog row, a renewable daily allowance, time-limited trial credits, and a paid API with signup credit are four different things. The free LLM API guide walks through how each provider meters it.

Batch and cache discounts change the ranking again

Google lists a Batch API at a 50% reduction for paid usage and a separate context-caching price per million tokens plus an hourly storage charge. If your workload tolerates delay or reuses a long prompt prefix, the cheapest option on the standard price list may not be the cheapest on the batch or cached price list. Interactive agents usually cannot wait for batch results, so check which column applies to you.

Quality reruns: where cheap gets expensive

The keep rate is where a cheap LLM API and a premium one diverge most, and it depends on the task. In Ficelle’s 76-task free-versus-paid benchmark, free routing tied or beat gpt-5 on 100% of classification tasks, 95% of JSON extraction tasks, and 94% of reasoning tasks, as judged by an LLM against per-task rubrics. On polished content writing, gpt-5 won 10 of the 20 tasks, so paid models won 50% of long-form writing. For customer-facing prose, the cheaper model’s keep rate drops, and a paid model or a human editor is the more economical choice.

Use that split as a starting hypothesis, then measure keep rates on your own prompts. A benchmark of 20 tasks per category is directional, not a guarantee for your workload.

How to compare cheap LLM APIs on your own workload

1. Pick three task types you run every day, for example ticket classification, field extraction, and code review comments.
2. Log real token counts for each type: input, visible output, and thinking tokens where the provider reports them.
3. Price each candidate at its current published rate, and note the date on the price page.
4. Count attempts per task over a day of real traffic, including 429 responses and timeouts.
5. Score a sample of answers with an objective check where you can, and record the keep rate.
6. Compute cost per accepted result with the formula above, then recheck when a provider changes prices or limits.

This takes an afternoon and replaces a single headline price with a figure that matches your bill.

Where Ficelle fits

If your comparison shows that classification, extraction, or routine reasoning works on free models, the remaining cost is keeping that free capacity usable. Free quotas run out, model IDs get retired, and a provider can start billing a model you thought was free.

Ficelle is a local, OpenAI-compatible router that sends requests to free models across several providers. It runs on your machine, keeps your keys local, and sends requests directly to the selected provider rather than through a Ficelle-hosted proxy. Strict-zero routing requires free pricing on the default path and keeps paid fallback disabled in code. When a model returns a rate limit, Ficelle applies a cooldown and can try another eligible candidate; when none can serve the request, it returns an error instead of spending money. Quota providers stay subject to their own account billing settings, so keep those accounts within their free allocation.

Ficelle does not make free models equal to paid ones, and it does not help with the writing tasks where gpt-5 won half the time. It is English-only and source-available under the Business Source License 1.1. Ficelle Core ships with OpenRouter and Nous as reference providers; Pro adds the maintained provider pack, including Groq and Gemini, with updates when a provider changes its catalog or limits.

If your numbers say most of your traffic belongs on free models and you would rather not maintain the provider list yourself, compare Ficelle plans.