undefined | Better HN

0 pointsrobbomacrae1y ago0 comments

Disturbing indeed. I've been worried a push back in AI is coming and this sort of story could be a tipping point and certainly would justify a period of reflection.

And your probably right that the alternatives maybe worse, the folks behind Lavender could probably even prove it with data.. but there should be a moral impetus to always have a human in the loop regardless. And any such attempt to justify it won't capture the publics attention like a sky-net doomsday happening over the civilians in Gaza.

0 comments

wruza1y ago

there should be a moral impetus to always have a human in the loop regardless

I don’t understand how to come to this. War is crap, not a dinner party. There’s always a human on both sides who will drop a bomb and laugh on camera, with no responsibility. Go watch it (actually don’t, it’s NSFL). Reading this thread feels like everyone watched and believed in that movie where they tried to select and eliminate a target for 2 hours with futuristic hi-tech. A human hesitates to press the button before the war. When in it, he will only be concerned with things like ammunition saving and tactical nuances. There’s not much more morals in a human who usually sits there at the button than in AI automation.

outside12341y ago

The thing that is different is now that human has an excuse: "The computer told me to put them in the oven."

NoGravitas1y ago

There's an old IBM presentation going around, from 1979, that says "A computer can never be held accountable, therefore a computer must never make a Management Decision." We know that humans make monstrous decisions in war; many of us remember seeing the Collateral Murder video, and everyone has at least heard of the Nuremberg trials. When humans make monstrous decisions, at least some of them, sometimes, hang for it. The computer here serves mainly to diffuse responsibility for decisions that would be made in any case. Who will hang?

robbomacraeOP1y ago

It's a good question. I also immediately found myself asking the same one of myself after posting that comment. I guess part of me just wants as many possible breakpoints along the process as possible.

But also at least then you have someone who is liable when things go wrong. When its fully automated, like the other comment mentions, they can just shrug and blame the AI. Who gets sued when a self driving car kills someone by accident? I don't know. Perhaps a lack of ownership is excusable. But when a weapon deliberately kills someone I think we need to have ownership somewhere.

Perhaps as a general rule the maker of the AI system should have liability for the AI up until someone else signs and accepts that responsibility. None of this "Company does not accept liability" crap. They have to make it clear that "customer accepts liability" or else it's them. That way they will be incentivized to make the military or whoever sign.

C6JEsQeQa5fCjE1y ago

I would strongly argue that even being able to prove that a human might perform worse is not an acceptable excuse for the reasons I will outline. The bar for a computer needs to be significally higher than that of a human.

We know that humans can make mistakes, due to a multitude of reasons. They can be tired, moody, distracted, stressed out, time-pressured, simply not care enough, etc, all contributing to making the wrong call. But a computer does not suffer from such issues. Secondly, a computer (program) is able to perform billions and billions of computations within some time period in order to ENSURE that doing this thing with grave consequences is absolutely warranted.

Maybe for some domains we can tolerate errors from AI, but when deciding whether a person (and everyone around them) lives or dies, surely simply being on average even more accurate than a human is not enough. "Killbots" MUST be extremely heavily regulated.

coffeebeqn1y ago

Pushback on AI will of course have a “National security” exception. If the industrial level facial recognition tech in Xinjiang was forgotten I doubt this will make a difference

j / k navigate · click thread line to collapse