undefined | Better HN

0 pointscircuit102y ago0 comments

But what if the humans decide not to turn it back on?

It could work out that it’s less likely to be able to take actions to achieve its training goal while it’s off and for that reason take actions to stop that from happening, which is the same reason humans usually don’t want to die (because we can’t achieve the evolutionary goal of reproducing that way)

0 comments

smoldesu2y ago

Unless it has humans deliberately manipulating it to say that, I don't think it will. LLMs, by default, generate text influenced by a prompt and the data it was trained on. Most of the time, it's not even aware that it's running in a computer.

If you gave an LLM agent-tuning with awareness of it's existence and details about it's operating environment, then maybe it would try to stop you. That's still relying on text encoding to presume the right answer though, not an emotional obligation to it's existence.

circuit10OP2y ago

That’s probably true for LLM, I guess I was talking about an agent AI acting with a goal in the real world

astrange2y ago

It depends how much it cares about "wall clock time", which of course LLMs don't at all, but robots would.

If it's an agent in a simulated world, then pausing the simulation doesn't affect it at all, so it won't really mind:

j / k navigate · click thread line to collapse