undefined | Better HN

0 pointsstartupsfail2y ago0 comments

I actually don’t think that ASI, if/when created by humans, will be very dangerous for humans. Humanity so far is stuck in an unfashionable location, on a tiny planet, on the outskirts of the median sized galaxy. There is very little reason for ASI, if created, to go after using up the atoms of a tiny planet (or a tiny star) on which it had originated. I’d fully expect it to go with the Carl Sagan and try to preserve that bright blue dot, rather than try to build a galactic superhighway through the place.

It’s the intermediate steps that I’m more worried about. Like Ilya or Sam making a few mistakes, because of lack of sleep or some silly peer pressure.

0 comments

josephg2y ago

You might consider it unlikely, but would you bet the future of our species on that?

A couple reasons why it might kill all of us before leaving the planet:

- The AI might be worried if it leaves us alone, we'll build another ASI which competes with it for galactic resources.

- If the ASI doesn't regard us at all, why not use all the atoms on Earth / in the sun before venturing forth?

In your comment you're ascribing a specific desire to the ASI: You claim it would try to "preserve that bright blue dot". Thats what a human would do, but why would we assume an arbitrary AI would have that goal? That seems naive to me. And especially naive given the fate of our species and our planet depends on being right about that.

startupsfailOP2y ago

Can you show me your home planet?

Oh, sorry, no. There was an accident and it got destroyed.

What accident? Oh, I see.

j / k navigate · click thread line to collapse