Actually Tested· Part 4
Stop asking AI for the news: 45% of answers had a serious problem.
45% of AI assistants' news answers had at least one significant issue, graded by journalists at 22 broadcasters.
Research
The research behind the hype.
Published experiments, benchmarks and real bills — read, checked number by number and explained without jargon. We don’t run the experiments. We read them so you don’t have to, and show every source.
- Actually Tested
Real tests. Real takeaways.
Published experiments and benchmarks — read, checked and explained. We don’t run them.
4 studies
- Real Build Cost
What it really cost.
What software and AI really cost to run, from published bills and breakdowns.
4 breakdowns
Latest research
- Real Build Cost· Part 4
Swap your AI model: the same answer costs 1,000× less than it did in 2021.
The same AI answer costs 1,000× less than in 2021: $60 per million tokens became $0.06.
- Actually Tested· Part 3
Stop underestimating ChatGPT: it beat 50 doctors in a real test. 92% vs 74%.
ChatGPT beat 50 doctors in a real diagnosis test, 92% to 74%.
- Real Build Cost· Part 3
Add a spending cap: a free website got a $104,500 bill for bandwidth.
A free Netlify site was billed $104,500 for bandwidth after a traffic spike; the bill was dropped only after the story hit Hacker News.
- Actually Tested· Part 2
Keep your job: the best AI agent finished 30% of the office work.
The best AI agent finished 30% of the tasks at a simulated software company.
- Real Build Cost· Part 2
Build a top AI model for $5.6M. DeepSeek did it. GPT-4 cost about $78M.
DeepSeek-V3's final training run cost $5.6M in rented GPU time, against an estimated $78M for GPT-4.
- Actually Tested· Part 1
Stop trusting the hype: AI made expert coders 19% slower. They felt 20% faster.
AI made experienced coders 19% slower in a real timed test, while they believed they were 20% faster.
- Real Build Cost· Part 1
Buy your own servers: Basecamp's cloud bill fell from $3.2M to $1.3M a year.
Basecamp's cloud bill fell from $3.2M a year to $1.3M after moving onto its own Dell hardware.
