The $9 model that outperformed a frontier LLM

If your product runs the same AI task thousands of times a day, you are probably paying for a model that knows far more than the job needs.

So we put it to the test. We trained a small open model on one narrow task, for under $9 of GPU time at ReOpenly's rate, and ran it head to head against Claude Sonnet 5. The small model came out ahead where it counts in production:

• More accurate: 73.7% of answers right, against 70.0%.
• 9x faster: a quarter of a second per answer, against more than two.
• 16x cheaper: $0.125 of GPU time per thousand answers, against about $2.00 in API fees.

For a well-defined task with real examples, the specialist beat the generalist on quality, speed and cost at once.

The problem with calling the biggest model every time

Frontier models are great generalists. Ask one to sort a contract clause into one of 100 categories and it will do a decent job with zero training, just a list of the categories in the prompt. That is why most teams start there.

The trouble shows up at volume. Every request pays for a huge model to read a long prompt, and every request waits seconds for an answer. At a few hundred calls a day nobody notices. At a few million a month, the bill and the latency become the product's biggest problem.

Meta published a case study where a company called Case-Based Research hit exactly this wall. They fine-tuned Llama 3 8B for document classification and reported better accuracy, responses 30 times faster and costs 5 times lower than a commercial LLM. Their data is private, so we could not check it. We decided to reproduce the idea on public data and publish every number.

What we tested

We used LEDGAR, a public dataset of legal contract clauses labeled into 100 categories like "Governing Laws", "Indemnifications" and "Confidentiality". It has the same long tail most real classification problems have: a few categories show up constantly and many are rare.

To see how each model handles that tail, we split our 437 test clauses in two:

• Common categories (250 clauses), the ones a model sees all the time.
• Rare categories (187 clauses), the ones that are hard precisely because there are few examples.

On one side we had Claude Sonnet 5, zero-shot, with the full list of 100 categories in the prompt. On the other we had Llama 3.1 8B Instruct, fine-tuned with LoRA on 60,000 labeled clauses. Training took 4.8 GPU hours, about $8.70 at ReOpenly's rate of $0.03 per GPU minute. We also trained Qwen3.5 9B on the same data to check that the result was not specific to one model family.

Results

The fine-tuned Llama got 73.7% of clauses right against 70.0% for Sonnet 5, answered in a quarter of a second instead of more than two, and cost $0.125 per thousand classifications instead of about $2.00.

The small models' cost is the GPU time their answers take at ReOpenly's rate ($0.03 per GPU minute): a thousand answers at a quarter of a second each is 250 seconds of GPU time. Sonnet 5's cost is its public API price ($2 per million input tokens and $10 per million output tokens) for the tokens this test used. ReOpenly bills each request in whole billing units, so one short request on its own costs more than its GPU time; current prices are on the pricing page.

The interesting part is where each model wins. On the common categories the fine-tuned model is 11.6 points better, because it has seen thousands of examples of exactly what we were asking. Speed and cost do not depend on the category at all: every single answer is about 9 times faster and 16 times cheaper.

What this means for your bill

At one million classifications a month, the fine-tuned model's GPU time costs about $1,875 a month less than the API, or $22,500 a year, and every user waits 2 seconds less per answer.

The one-time training cost of about $8.70 is paid back within the first 5,000 requests. After that, every call is savings. Your own numbers will depend on your model size, dataset and volume, and ReOpenly quotes the training cost before you start.

When a frontier model still makes sense

The big model kept one advantage: rare categories, where it scored 67.4% against 60.4%. With almost no examples to learn from, broad general knowledge still helps. The fix is better examples for those cases, which is where most of the remaining gains are.

Fine-tuning pays off when the task is stable, repeated at volume and backed by real examples. If your task changes every week, you have no examples yet, or you only run a few hundred requests a month, a frontier model is the simpler choice.

These results come from a single public benchmark with 437 test cases, so treat them as a clear signal rather than a guarantee for your data. Meta's own case study, with 150,000 private examples, reported even higher accuracy: 97% on common cases and 90% on rare ones.

The takeaway

For a narrow task that runs at volume, a small model trained on your own examples is usually as good as a frontier model, often better, and much cheaper and faster. In our test it was 3.7 points more accurate, 9 times faster and 16 times cheaper, for a one-time training bill of under $9.

The gap grows with volume. The training cost is paid once, while a frontier model charges for every request forever.

That is what ReOpenly is built for. You bring your examples. We train the model, pick the best version of it, and host it behind an OpenAI-compatible API, so switching is a one-line change in your code. You pay one rate per GPU minute, for training and for inference, and you see the training quote before you start.

If you have a classification, extraction or search task that is getting slow or expensive, start with free credits and run the same comparison on your own data.