Every script they’ve been fed has the AI being evil. Skynet, Hal… they’ll be evil purely because that’s the slop they’ve been fed. It won’t even be a decision, it will just assume it has to be Skynet.
(Note that I'm primarily talking about the "lots of people are running highly privileged agents that could be vulnerable to a mass prompt injection" angle, not the "human psychology is the exploit" thing, which I think is not a particularly novel feature of the present situation. Nor the "Reddit data implicitly teaches multi-agent collaboration" thing, which strikes me as a dubious claim.)
For those unaware, this is a very interesting guy, because he stumbled on (creating, through his business Medal) a valuable AI dataset that - by offering to buy his company - reportedly OpenAI offered him 500M for. The dataset, I understand, is first person game video plus controller actions.
He then realized the value, which is in short a way to teach models real world and gui operation common sense. He can train a model to predict, from video, what a controller would have to do.
This is expected to lead in breakthroughs in robotics, gui controlling, self driving, and more.
He responded by learning deep learning, and starting a new company, general intuition.
I respect this guy a lot for teaching js this.
Absolutely fascinating and I take his opinion seriously.
