undefined | Better HN

0 pointsTazeTSchnitzel11y ago0 comments

The Internet Archive should implement some sort of digital signature system to allow website owners with foresight to prevent this.

0 comments

toyg11y ago

They could just timestamp different versions of robots.txt (which they probably do already), and respect it depending on date (which is more of a hassle, because you have to build it in your UI logic).

nathanb11y ago

That would not solve the problem they're trying to solve.

Let's say I post something that I shouldn't have posted -- insider stock information, nude photos, whatever. Perhaps something illegal for me to post. I need to make it go away.

I need to be able to create a robots.txt today which affects stuff I posted yesterday.

This is why archive.org respects the current robots.txt for access to past content.

j / k navigate · click thread line to collapse