undefined | Better HN

0 pointsdefen6y ago0 comments

> Why, in principle, would it not be possible for us to design an AGI, that would have care for our (all sentient beings') welfare or care for the investors' profit as (one of) its core goal(s)?

Because we don't know how to design goal functions. Furthermore, how would the AI measure "welfare"? Maybe the way it maximizes welfare is horrifying to us. Look at how easy it is to hack current image recognition neural nets, then imagine a solution to the human welfare problem that is as far from an image of a dog as an image of pink noise is.

0 comments

DuskStar6y ago

> Because we don't know how to design goal functions.

IIRC that's a large part of what OpenAI's trying to solve. But it is a very hard problem.

I've heard a 'joke' before that there are three kinds of Genies (or AIs) - ones where you can wish for what you should wish for, ones where wishing for anything results in horrible outcomes, and ones that aren't interesting. The goal of OpenAI isn't just to make strong general AI - it's also to make sure it falls into the first category and not the second.

j / k navigate · click thread line to collapse

0 pointsdefen6y ago0 comments

> Why, in principle, would it not be possible for us to design an AGI, that would have care for our (all sentient beings') welfare or care for the investors' profit as (one of) its core goal(s)?

0 comments

DuskStar6y ago

> Because we don't know how to design goal functions.

IIRC that's a large part of what OpenAI's trying to solve. But it is a very hard problem.

j / k navigate · click thread line to collapse