when Claude Opus 5 was run at Max reasoning effort, it scores 97.5% on ARC-AGI-1 and 90.4% on ARC-AGI-2 semi-private
被忽略的第三个自由变量:推理预算。30.2% 那条基线是 high effort,而 AVO 的跑法用了另一套 reasoning setting。同一模型换档位分数就能大幅移动,所以任何跨系统比分表,都得先标注 effort 档位、观测格式和评测集,再谈差值。
when Claude Opus 5 was run at Max reasoning effort, it scores 97.5% on ARC-AGI-1 and 90.4% on ARC-AGI-2 semi-private
被忽略的第三个自由变量:推理预算。30.2% 那条基线是 high effort,而 AVO 的跑法用了另一套 reasoning setting。同一模型换档位分数就能大幅移动,所以任何跨系统比分表,都得先标注 effort 档位、观测格式和评测集,再谈差值。