undefined | Better HN

0 pointsprismlfx3d ago0 comments

Where did you see the big crawlers don't read it? Anthropic does.. they're pretty big.

0 comments

If I look at the logs of sites that Anthropic crawls (and in some cases references in answers) and there are no requests for llms.txt

j / k navigate · click thread line to collapse

If I look at the logs of sites that Anthropic crawls (and in some cases references in answers) and there are no requests for llms.txt

j / k navigate · click thread line to collapse