如今，仅凭通过/不通过的成绩来解读评估几乎是不可能的，我看到的许多基准测试中的失败案例，往往是因为……
it's basically impossible to interpret evals by looking at just at the pass/fail scores these days many of the failures I see in benchmarks are due to...
ThariqAI2026-09-11
如今仅凭通过/不通过的成绩来解读评估几乎是不可能的。我看到的许多基准测试失败案例，都是由于过于严格的隐藏测试，在某些情况下，模型的答案甚至比预期的评估结果更有道理。
原文
it's basically impossible to interpret evals by looking at just at the pass/fail scores these daysmany of the failures I see in benchmarks are due to overly strict hidden tests, in some cases the model's answer makes more sense than the expected eval result