Each task includes a unified evaluation framework supporting sandboxed code and APIs, alongside a human reference trajectory annotated with stepwise checkpoints along dual-axis: S-axis and V-axis.
大多数人认为AI评估可以通过简单的自动化测试完成。但作者提出需要复杂的双轴(S-axis和V-axis)人工参考轨迹和沙箱环境支持，这暗示了评估AI代理能力的极端复杂性远超当前行业的普遍认知。这一观点挑战了AI评估的简化主义倾向，强调了人类参与在评估中的不可替代性。