computer-use agents extend language models from text generation to persistent action over tools, files, and execution environments
主流观点认为文本语言模型和计算机使用代理的安全挑战本质上是相同的,只需将文本安全措施扩展即可。但作者指出,计算机使用代理引入了持久状态、工具使用和执行环境等全新维度,创造了与纯文本系统完全不同的安全挑战,这挑战了简单的安全扩展假设。
computer-use agents extend language models from text generation to persistent action over tools, files, and execution environments
主流观点认为文本语言模型和计算机使用代理的安全挑战本质上是相同的,只需将文本安全措施扩展即可。但作者指出,计算机使用代理引入了持久状态、工具使用和执行环境等全新维度,创造了与纯文本系统完全不同的安全挑战,这挑战了简单的安全扩展假设。
Hallucinated packages are the sleeper threat. LLMs regularly invent package names that don't exist. One study found that nearly 20% of AI-recommended packages were fabrications, and 43% of those hallucinated names appeared consistently across queries.
大多数人认为AI推荐的包都是真实存在的,但作者揭示了AI经常推荐不存在的包,这已成为一种新的攻击向量。攻击者利用这一现象注册'幻觉包'并植入恶意代码,这种'slopsquatting'技术让AI本身成为供应链攻击的放大器。
To this day, if you know the right people, the Silicon Valley gossip mill is a surprisingly reliable source of information if you want to anticipate the next beat in frontier AI – and that’s a problem. You can’t have your most critical national security technology built in labs that are almost certainly CCP-penetrated
for - high security risk - US AI labs
for - progress trap - AI superintelligence - interview - AI safety researcher and director of the Cyber Security Laboratory at the University of Louisville - Roman Yampolskiy - progress trap - over 99% chance AI superintelligence arriving as early as 2027 will destroy humanity - article UofL - Q&A: UofL AI safety expert says artificial superintelligence could harm humanity - 2024, July 15
this company's got not good for safety
for - AI - security - Open AI - examples of poor security - high risk for humanity
AI - security - Open AI - examples of poor security - high risk for humanity - ex-employees report very inadequate security protocols - employees have had screenshots capture while at cafes outside of Open AI offices - People like Jimmy Apple report future releases on twitter before Open AI does
open AI literally yesterday published securing research infrastructure for advanced AI
for - AI - Security - Open AI statement in response to this essay
this is a serious problem because all they need to do is automate AI research 00:41:53 build super intelligence and any lead that the US had would vanish the power dynamics would shift immediately
for - AI - security risk - once automated AI research is known, bad actors can easily build superintelligence
AI - security risk - once automated AI research is known, bad actors can easily build superintelligence - Any lead that the US had would immediately vanish.
the model Waits are just a large files of numbers on a server and these can be easily stolen all it takes is an adversary to match your trillions 00:41:14 of dollars and your smartest minds of Decades of work just to steal this file
for - AI - security risk - model weight files - are a key leverage point
AI - security risk - model weight files - are a key leverage point for bad actors - These files are critical national security data that represent huge amounts of investment in time and research and they are just a file so can be easily stolen.
our failure today will be irreversible soon in the next 12 to 24 months we will leak key AGI breakthroughs to the CCP it will 00:38:56 be to the National security establishment the greatest regret before the decade is out
for - AI - security risk - next 1 to 2 years is vulnerable time to keep AI secrets out of hands of authoritarian regimes
here are so many loopholes in our current top AI Labs that we could literally have people who are infiltrating these companies and there's no way to even know what's going on because we don't have any true security 00:37:41 protocols and the problem is is that it's not being treated as seriously as it is
for - key insight - low security at top AI labs - high risk of information theft ending up in wrong hands
Openai is looking to predict performance and safety because models are too big to be evaluated directly. To me this implies a high probability that people start to replace their own capabilities with models not enough safe and relevant. It could cause misalignment between people and their environment, or worse their perception of their environment.
Seeing how powerful AI can be for cracking passwords is a good reminder to not only make sure you‘re using strong passwords but also check:↳ You‘re using 2FA/MFA (non-SMS-based whenever possible) You‘re not re-using passwords across accounts Use auto-generated passwords when possible Update passwords regularly, especially for sensitive accounts Refrain from using public WiFi, especially for banking and similar accounts
看到人工智能在破解密码方面有多么强大,这很好地提醒了我们,不仅要确保你在使用强密码,还要检查:
你正在使用 2FA/MFA(尽可能不使用基于短信的)。
你没有在不同的账户间重复使用密码
尽可能使用自动生成的密码
定期更新密码,特别是敏感账户的密码
避免使用公共WiFi,尤其是银行和类似账户
Now Home Security Heroes has published a study showing how scary powerful the latest generative AI is at cracking passwords. The company used the new password cracker PassGAN (password generative adversarial network) to process a list of over 15,000,000 credentials from the Rockyou dataset and the results were wild. 51% of all common passwords were cracked in less than one minute, 65% in less than an hour, 71% in less than a day, and 81% in less than a month.
Centre for Effective Altruism. (2020, June 13 & 14). EAGxVirtual 2020 Virtual Conference. https://www.youtube.com/playlist?list=PLwp9xeoX5p8NfF4UmWcwV0fQlSU_zpHqc
Enligt Polismyndighetens riktlinjer ska en konsekvensbedömning göras innan nya polisiära verktyg införs, om de innebär en känslig personuppgiftbehandling. Någon sådan har inte gjorts för det aktuella verktyget.
Swedish police have used Clearview AI without any 'consequence judgement' having been performed.
In other words, Swedish police have used a facial-recognition system without being allowed to do so.
This is a clear breach of human rights.
Swedish police has lied about this, as reported by Dagens Nyheter.