Skip to main content

One post tagged with "evaluation"

View all tags

[LLM 9/10] Benchmarking: An Accuracy Number Without a Confidence Interval Is a Rumor

· 25 min read
Kobkrit Viriyayudhakorn
CEO, iApp Technology

Since chapter 1, this series has repeated the same sentence every time it reports a number: "accuracy without a confidence interval is not an experimental result, it's a rumor" — and promised a full explanation in chapter 9. This is that chapter. We won't train a single step. Instead we'll build a three-mode evaluation system from scratch, put every checkpoint trained across the series on real Thai exams, and prove that the settings nobody writes down in papers move the very same model's score by more than the leaderboard gaps people argue about.