Swap your AI model: the same answer costs 1,000× less than it did in 2021.
The same AI answer costs 1,000× less than in 2021: $60 per million tokens became $0.06.
We didn’t run this. a16z did. ModelSignal read the published work and checked every number below against it — the sources and the dates we read them are at the end.
Same quality · new price
$0.06
per million tokens.
It was $60 in 2021.
a16z, November 2024. GPT-3-level quality.
2021 vs 2024
GPT-3
$60
Per million tokens
Who could do itOnly GPT-3
Llama 3.2 3B
$0.06
Per million tokens
Same test score42
Re-price it.
Same quality, 1,000× cheaper in three years. a16z's rule: 10× cheaper every year.
a16z · LLMflation · 12 Nov 2024
The rule of thumb
10×
cheaper every year, for
the same quality.
a16z's rule of thumb. 1,000× in three years.
The proof
a16z's LLMflation post, 12 Nov 2024. Read 13 Oct 2026.
The full breakdown
Somebody else did this math; we read it. Every number below is quoted from the a16z post, with the date we read it.
The post
Guido Appenzeller, Welcome to LLMflation – LLM inference cost is going down fast, a16z, 12 November 2024.
The numbers
| GPT-3, November 2021 | "the only model that was able to achieve an MMLU of 42 — at a cost of $60 per million tokens" |
| Cheapest at the same score, Nov 2024 | "Llama 3.2 3B, from model-as-a-service provider Together.ai, at $0.06 per million tokens" |
| Overall | "The cost of LLM inference has dropped by a factor of 1,000 in 3 years." |
| Rule | "For an LLM of equivalent performance, the cost is decreasing by 10x every year." |
| GPT-4 level (MMLU 83) | "the price for models at this level have come down by about a factor of 62" since March 2023 |
The six drivers, per a16z
Better cost/performance of the GPUs · Model quantization · Software optimizations · Smaller models · Better instruction tuning · Open source
Source
- https://a16z.com/llmflation-llm-inference-cost/ (read 13 Oct 2026)
Sources
The catches
- One benchmark, MMLU, is the yardstick for 'equivalent performance'; other tasks may not track it.
- Prices are list prices from a few providers at the time of writing; the post is a trend line, not a guarantee.
- The post is by a venture firm with portfolio interests in the sector; the figures are checkable but the framing is theirs.
Every number, and where it’s quoted from
- cost falls 10x every year for equivalent performance“For an LLM of equivalent performance, the cost is decreasing by 10x every year.”a16z LLMflation, 12 Nov 2024
- GPT-3, November 2021, MMLU 42, $60 per million tokens“When GPT-3 became publicly accessible in November 2021, it was the only model that was able to achieve an MMLU of 42 — at a cost of $60 per million tokens.”a16z LLMflation
- Llama 3.2 3B via Together.ai at $0.06 per million tokens, same score, as of writing (Nov 2024)“As of the time of writing, the cheapest model to achieve the same score was Llama 3.2 3B, from model-as-a-service provider Together.ai, at $0.06 per million tokens.”a16z LLMflation
- factor of 1,000 in 3 years“The cost of LLM inference has dropped by a factor of 1,000 in 3 years.”a16z LLMflation
- MMLU 83 (GPT-4 level) down by about a factor of 62 since March 2023“If we pick a higher MMLU score of 83, we have less data because models of this quality level have only existed since GPT-4 came out in March of 2023. Since then, however, the price for models at this level have come down by about a factor of 62.”a16z LLMflation
- six drivers: GPUs, quantization, software, smaller models, instruction tuning, open source“Better cost/performance of the GPUs · Model quantization · Software optimizations · Smaller models · Better instruction tuning · Open source”a16z LLMflation, section headings
- author Guido Appenzeller, a16z, 12 November 2024“Guido Appenzeller — November 12, 2024”a16z
