The process will never be fully automated, regardless of what they say. Cases will need to be reviewed. Things will need to be checked at some point.

They are trying to play the cog in the machine, that mechanically transmits information to law enforcement. But if we’ve learnt anything the last decade is that cogs are not impartial and can be very dangerous, if only because of the scale at which they operate. I can see several ways a user can face a kafkaesque uphill battle to prove their innocence. In several countries, just a child pornography case can be a social death sentence. And even a fraction of a percent of mistakes will mean millions of people might be dragged in this (let’s not kid ourselves: this is never going to stay in the US).

Personally I think (what Apple is doing) is misguided and ripe for abuse. I am very disappointed that they, of all companies, are pushing this nightmare.

> As to who decides what constitutes significance? That is where you'll hit the most problems, and reasonable discussion of it will be quickly shut down with the same arguments used for automating a flagging system. The conversation requires nuance, but those currently calling for such systems aren't interested in a good faith discussion.

Ultimately, a tech company has no business making this sort of decisions. This is something that needs to be sorted out by law.

Unfortunately, a nuanced discussion is very unlikely these days. Anyone looking not agressive enough will be pilloried.