The model saw the same case and the same 19 labels; only their positions changed. Yet models frequently changed answers!
只换选项顺序、病例和标签一字不动,模型答案就变,说明分数里混着大量与医学无关的形式噪声。这条可以直接迁移到任何榜单:看到排名差几个百分点,先问是不是换个 prompt、换个选项顺序就翻盘,别急着当能力差距读。
The model saw the same case and the same 19 labels; only their positions changed. Yet models frequently changed answers!
只换选项顺序、病例和标签一字不动,模型答案就变,说明分数里混着大量与医学无关的形式噪声。这条可以直接迁移到任何榜单:看到排名差几个百分点,先问是不是换个 prompt、换个选项顺序就翻盘,别急着当能力差距读。