undefined | Better HN

0 pointsthreeseed1y ago0 comments

> And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement where they've already pirated shittons of data

This is wrong. AppleBot identifier hasn't changed: https://support.apple.com/en-us/119829

There is no AppleBot-Extended. And if you blocked it in the past it remains blocked.

0 comments

fotta1y ago

From your own link:

> Controlling data usage

> In addition to following all robots.txt rules and directives, Apple has a secondary user agent, Applebot-Extended, that gives web publishers additional controls over how their website content can be used by Apple.

> With Applebot-Extended, web publishers can choose to opt out of their website content being used to train Apple’s foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.

ziml771y ago

But it also says that Applebot-Extended doesn't crawl webpages and instead this marker is only used to determine what can be done with the pages that were visited by Applebot.

Not that I like an opt-out system, but based on the wording of the docs it is true that if you blocked Applebot then blocking Applebot-Extended isn't necessary.

fotta1y ago

Yeah that is true, but I suspect that most publishers that want their content to appear in search but not used for model training will not have blocked Applebot to date (hence the original commenter's argument)

threeseedOP1y ago

Might want to actually read it:

Applebot-Extended does not crawl webpages.

They gave this as an additional control to allow crawling for search but blocking for use in models.

fotta1y ago

> There is no AppleBot-Extended. And if you blocked it in the past it remains blocked.

You said there is no Applebot-Extended. The link says otherwise.

1 more reply

j / k navigate · click thread line to collapse

0 comments

fotta1y ago

From your own link:

> Controlling data usage

ziml771y ago

But it also says that Applebot-Extended doesn't crawl webpages and instead this marker is only used to determine what can be done with the pages that were visited by Applebot.

Not that I like an opt-out system, but based on the wording of the docs it is true that if you blocked Applebot then blocking Applebot-Extended isn't necessary.

fotta1y ago

threeseedOP1y ago

Might want to actually read it:

Applebot-Extended does not crawl webpages.

They gave this as an additional control to allow crawling for search but blocking for use in models.

fotta1y ago

> There is no AppleBot-Extended. And if you blocked it in the past it remains blocked.

You said there is no Applebot-Extended. The link says otherwise.

1 more reply

j / k navigate · click thread line to collapse