Center for AI Safety, Scale AI Launch 2,500-Question AI Benchmark as Top Models Score 45%-50%
Updated
Updated · KDnuggets · Jul 2
Center for AI Safety, Scale AI Launch 2,500-Question AI Benchmark as Top Models Score 45%-50%
3 articles · Updated · KDnuggets · Jul 2
Summary
More than 2,500 expert-level questions across 100-plus disciplines make up Humanity’s Last Exam, a new benchmark built by the Center for AI Safety and Scale AI to test advanced AI reasoning.
45%-50% is the rough score range for leading models such as GPT, Gemini and Claude, underscoring the test’s difficulty and exposing overconfident wrong answers rather than simple recall failures.
About 60% of expert views surveyed in the report see HLE as useful because older benchmarks like MMLU have been saturated, with modern systems often scoring above 90% and becoming hard to distinguish.
Roughly 30% call the benchmark a distraction from real-world AI performance, while a smaller group argues some answer keys are flawed, especially in niche chemistry and advanced math questions.
Published in Nature in January 2026, HLE is widely viewed as a tougher way to compare frontier models, even as critics dismiss its “Humanity’s Last Exam” branding as marketing.