90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity
这两个百分比数据点(90.6%验证率，62.4%确认高危率)对于评估AI模型在安全漏洞检测中的可靠性至关重要。90.6%的验证率表明AI模型的误报率相对较低，这在AI安全领域是相当出色的表现。然而，62.4%的确认高危率意味着近40%的AI评估高危漏洞实际严重程度较低，这反映了AI在严重性评估上仍有改进空间。
_Self-reported score with custom Anthropic scaffold._ SWEPro were evaluated with the mini-swe-agent scaffold. However, we use the scores reported by Anthropic for Opus with the max thinking efforts due to frequent timeouts during our evaluation trials.
脚注2揭示了重要数据点：Opus 4.6的53.4分是Anthropic的自报分数，因为作者在评估过程中频繁遇到超时问题，无法自行验证。这表明性能比较中存在数据可靠性问题，特别是对于Opus的评估依赖于厂商自报数据，可能存在偏差。
Argument quality and fallacies. (n.d.). HackMD. Retrieved January 17, 2022, from https://hackmd.io/@scibehC19vax/argumentquality
Scientists try to pinpoint why rapid Covid tests are missing some cases. (2022, January 6). STAT. https://www.statnews.com/2022/01/06/scientists-try-to-pinpoint-why-rapid-covid-tests-are-missing-cases/
Taylor, L. (2021). The Venezuelan health-care workers secretly collecting COVID stats. Nature, 597(7874), 20–21. https://doi.org/10.1038/d41586-021-02276-1
I haven't met anyone who makes this argument who then says that a one stop convenient, reliable, private and secure online learning environment can’t be achieved using common every day online systems
Reliable: As a simple example, I'd trust Google to maintain data reliability over my institutional IT support.
And you'd also need to make the argument for why learning needs to be "private", etc.
Speelman, C., & McGann, M. (2020). Statements about the Pervasiveness of Behaviour Require Data about the Pervasiveness of Behaviour [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/bxzm4
Parsons, Sam. ‘Reliability Multiverse’, 26 June 2020. https://doi.org/10.31234/osf.io/y6tcz.
DaveLagnado on Reddit
Fenton, N., Hitman, G. A., Neil, M., Osman, M., & McLachlan, S. (2020). Causal explanations, error rates, and human judgment biases missing from the COVID-19 narrative and statistics [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/p39a4
Edelsbrunner, P. A., & Thurn, C. (2020, April 22). Improving the Utility of Non-Significant Results for Educational Research. https://doi.org/10.31234/osf.io/j93a2