undefined | Better HN

0 points0xDEAFBEAD2y ago0 comments

If you believe that aligned AGI will create a post-scarcity society, there's no reason for its creator to hoard the benefits for themselves. The main thing is to ensure that post-scarcity society actually comes into being.

Edit: Note also the "value-aligned, safety-conscious" caveat.

0 comments

cowl2y ago

there is no such thing as general post-scarcity though. there is post-scarcity of certain needs but not general post-scarcity. So it's creator would still have reason to hoard certain benefits for themselves while distributing some for all.

sudosysgen2y ago

It's generally meant that material scarcity becomes secondary to social wellbeing. Ie, material needs are so easy to take care of that the dynamics of social interaction undergo a fundamental shift, as they currently are strongly tied to material conditions.

0xDEAFBEADOP2y ago

Once you get past a certain level of wealth, the esteem of your fellow humans matters more than acquiring even more goods and services.

jacquesm2y ago

That's statistically speaking not very well supported. The most common element of those that are past a certain level of wealth is that they want more wealth, more power or both.

jacquesm2y ago

I don't believe that.

What I believe is that the first party to create an AGI will likely end up being nationalized and that that AI will be weaponized the next day to the exclusion of the rest of the world.

comp_throw72y ago

That's fine, but the OpenAI founders (including Sam Altman![1]) explicitly had in mind "AI kills everyone" as one of the main problems they were attempting to tackle when they started the organization. (Obviously starting OpenAI as a solution to this problem was a terrible mistake, but given that they did so, having a board charged with pulling the plug is better than not having that.)

[1] https://blog.samaltman.com/machine-intelligence-part-1

jacquesm2y ago

That whole charter is bullshit. You don't invite Microsoft to the party if you care about ethics and you don't disable your fail-safe by giving them full access to all of your stuff.

Hence my position that the charter was a fig leaf designed to throw regulators off.

1 more reply

j / k navigate · click thread line to collapse

0 comments

cowl2y ago

sudosysgen2y ago

0xDEAFBEADOP2y ago

Once you get past a certain level of wealth, the esteem of your fellow humans matters more than acquiring even more goods and services.

jacquesm2y ago

That's statistically speaking not very well supported. The most common element of those that are past a certain level of wealth is that they want more wealth, more power or both.

jacquesm2y ago

I don't believe that.

What I believe is that the first party to create an AGI will likely end up being nationalized and that that AI will be weaponized the next day to the exclusion of the rest of the world.

comp_throw72y ago

[1] https://blog.samaltman.com/machine-intelligence-part-1

jacquesm2y ago

That whole charter is bullshit. You don't invite Microsoft to the party if you care about ethics and you don't disable your fail-safe by giving them full access to all of your stuff.

Hence my position that the charter was a fig leaf designed to throw regulators off.

1 more reply

j / k navigate · click thread line to collapse