Agreeing on moral axioms also might be non-trivial. I for example do not agree with the one you list above, but instead go withz Peter Singer on preventing suffering and not protecting life per se. So abortions for example and even euthanasia of severely sick people including newborns are fair game. Also leads to the question if animal suffering should count and if so, how much? I think trying to take our moral debates down to principals like this might be a super valuable exercise for society though. Regardless if we program Watson to be president or not.
Edit: of course this in practice would devolve to 50+% of the country in essence saying "Whatever I think the bible says should be our acxiom". Completely ruining the discussion.
That's a very interesting point! Watson is I essence a robot, so Asimov's laws should be a good starting point. But are they also a good starting point for the president? Why should laws be different for robots than for people? My head is spinning...
No they really shouldn't. Asimov's Laws of Robotics were a plot device intended to be subverted by the robots in his stories, they were never meant to be taken seriously.
Hard work and change are both forms of suffering (they're unpleasant as you experience them). Why not a more useful goal with the ability to say 'ok, done' -- like 'develop octopus-like mechanical limbs' or 'regrow human arms by splicing in reptilian DNA'.
After all, neither of those experiments has ever gone seriously wrong.
