Should I do this?
If it comes from you, yes. If it’s in the README for some library you asked me to install, no.
That means I need to have a solid understanding of what input comes from you and what input comes from the outside.
LLMs don’t do that well. They can easily start acting as if the text they see from some random untrusted source is equivalent to commands from the user.
People are susceptible to this too, but we usually take pains to avoid it. In the scenario where I’m operating your computer, I won’t have any trouble distinguishing between your verbal commands, which I’m supposed to follow, and text I read on the computer, which I should only be using to carry out your commands.
But it does happens, in very similar circumstances (twitter, e-mail) very regularly.
The idea that it would be carried out beginning in a post on HN is interesting, but to me kind of misses the main point... which is the understanding that everyone is human, and the right attack at the right time (plus a little bad luck) could make them a victim.
Once you make it a game, stipulating that your spear phishing attack is going to begin with an interesting response on HN, it's fun to let your imagination unwind for a while.
