References

Funding

Acknowledgmen

That is a very dangerous place to be

likely

Conclusions and recommendations

analyze Apple’s recent CSAM proposal withrespect to safety and security

There’s an old adage: “in theory, there is no difference between theoryand practice, but in practice, there is.” This very much applies to CSS

ns [51,52], there has been a rapidly growing body of researchon the topic [53].In a nutshell, given access to models, or even just to the imagesor text used to train them, virtually any content can be tweakedto escape detection. In many circumstances, it is possible to createperturbations that evade any model [54]. Most of the proposed de-fenses have either been broken [55], or shown to impose a significantpenalty on the model’s performance [56]. Even more damning is in-creasing evidence that there are fundamental trade-offs that preventthe detection of all kinds of adversarial effects [57], and that beingable to detect adversarial inputs to filter them may be an unavoidablyhard problem [58]

False-positive attacks

et al

Even more, a malware or spyware app could intentionally puttargeted material onto a device secretly, knowing it will eventuallybe found. CSS assumes that the device is totally under its owner’scontrol, yet the situation is far more complex.

It can be argued

Apple deserves praise for do-ing this, and for delaying the launch of its system

Core policy principles

see the section on new secu-rity and privacy risks

Security and policy principles for contentscanning

a woman planning to escape a violent or controllingpartner who abuses both her and their children. The abuser will ofteninstall stalkerware on family phones, and he may use “smart home”infrastructure such as video doorbells as a way to control them [24]

antivirus software vendor Kaspersky might have a covert relation-ship with a national intelligence service [23]

the legal constraints against government action become weakerand the range of targeted content becomes larger, typically includ-ing LGBTQ+ content, political activists, and domestic rivals of au-thoritarian regimes[22]. In such places, CSS will provide a means ofrepression and political manipulation

local adversaries such as abusive partners or controllingfamily members

Inexamining the technical and societal risks of CSS, there are severaldistinct concerns: how to ensure that information within the systemis properly protected (a security concern); how to ensure that thesystem of central servers, human reviewers, user devices, users, andpotentially targeted content works appropriately (a socio-technicalconcern); and how to ensure that technologies with potential to be-come bulk-surveillance infrastructure can be deployed safely (a pol-icy concern).

To understand whether CSS can be justified as such, we need to lookat the nature of its actual and reasonably foreseeable intrusions indetail

spam detectorslearn to identify spam messages by relying on reports from millionsof email users clicking the “report spam” button to retrain the spamfilter models every day

The hash list is periodically updated by the curator

As we explained in the previous section

Theprovider first trains a machine-learning model with image sets con-taining both innocuous and target content. This model is then usedto scan pictures uploaded by users. Unlike perceptual hashing, whichdetects only photos that are similar to known target photos, machine-learning models can detect completely new images of the type onwhich they were trained. One well-known example is the face detec-tor used in iPhones to detect faces on which to focus the camera

Perceptual hashing

While the EuropeanUnion (EU) and Apple have been talking about child sex-abuse ma-terial (CSAM)—specifically images—in their push for CSS [12], theEU has included terrorism and organized crime along with sex abuse[13]. In the EU’s view, targeted content extends from still imagesthrough videos to text, as text can be used for both sexual solici-tation and terrorist recruitment. We cannot talk merely of “illegal”content, because proposed UK laws would require the blocking on-line of speech that is legal but that some actors find upsetting [14].

Content scanning technologies

Apple’s August 2021 proposal

ed content [3], andon the 2019 Carnegie Endowment for International Peace study onencryption policy, which presents a set of principles to guide solu-tions [4]. We also build on Paul Rosenzweig’s early analysis of thepolicy issues raised by CSS, along with some of the technical issues[5]. Since Apple announced its scanning proposal in August 2021,several researchers and organizations have provided rapid analysesof the proposal, and the technology and policy issues it raises. In par-ticular, we acknowledge Eric Rescorla of Mozilla [6], Kurt Opsahlof the Electronic Frontier Foundation [7], Steven Murdoch [8], PaulRosenzweig [9], and Daniel Kahn Gillmor of the American Civil Lib-erties Union (ACLU) [10]. Here, our aim is a more thorough technicalanalysis and to cover CSS more generally. In addition, our analysissheds some light on the design decisions that Apple took. Apple didits best, using some of the top talent in security and cryptography,and yet did not achieve a design for a secure, trustworthy, and effi-cacious system. Indeed, it appears that Apple has withdrawn fromimplementing the most ambitious aspects of its system [11]

We have built our study on the work of many others

Keywords: Client-side scanning; Risks; Perceptual hash

Abstract

Harold Abelson1 , Ross Anderson2,3 , Steven M. Bellovin4 ,∗,†,Josh Benaloh5 , Matt Blaze6 , Jon Callas7 , Whitfield Diffie8 ,‡ ,Susan Landau9 , Peter G. Neumann10 , Ronald L. Rivest1 , JeffreyI. Schiller1 , Bruce Schneier11,12 , Vanessa Teague13 , Carmela Troncoso14

Content scanning technologies

Apple’s August 2021 proposal

or technical approaches to access the unencrypted content [3], andon the 2019 Carnegie Endowment for International Peace study onencryption policy, which presents a set of principles to guide solu-tions [4]. We also build on Paul Rosenzweig’s early analysis of thepolicy issues raised by CSS, along with some of the technical issues[5]. Since Apple announced its scanning proposal in August 2021,several researchers and organizations have provided rapid analysesof the proposal, and the technology and policy issues it raises. In par-ticular, we acknowledge Eric Rescorla of Mozilla [6], Kurt Opsahlof the Electronic Frontier Foundation [7], Steven Murdoch [8], PaulRosenzweig [9], and Daniel Kahn Gillmor of the American Civil Lib-erties Union (ACLU) [10]. Here, our aim is a more thorough technicalanalysis and to cover CSS more generally. In addition, our analysissheds some light on the design decisions that Apple took. Apple didits best, using some of the top talent in security and cryptography,and yet did not achieve a design for a secure, trustworthy, and effi-cacious system. Indeed, it appears that Apple has withdrawn fromimplementing the most ambitious aspects of its system [11]

We have built our study on the work of many others

Keywords: Client-side scanning; Risks; Perceptual hash

Abstract

Harold Abelson1 , Ross Anderson2,3 , Steven M. Bellovin4 ,∗,†,Josh Benaloh5 , Matt Blaze6 , Jon Callas7 , Whitfield Diffie8 ,‡ ,Susan Landau9 , Peter G. Neumann10 , Ronald L. Rivest1 , JeffreyI. Schiller1 , Bruce Schneier11,12 , Vanessa Teague13 , Carmela Troncoso

Yaman Roumani a,*, Mais Alraee ba Department of Decision and Information Sciences, Oakland University, Elliott Hall, Room 427, 275 Varner Drive, Rochester, MI, 48309-4485b Department of Mathematics Statistics and Data Science, Montgomery College, 20200 Observation Drive, Germantown, MD, 20876