Genuine question: is there a big inherent difference between "I don't understand this thing but I think this other human does," and "I don't understand this but I think this other AI does"?
If your answer is "yes," do you think that's inherent to the (metaphysical?) fact of it being AI or to specific limitations to current AI? If the latter, what changes to AI would let you trust it?
I don't know. AI has an understand of some really complex things, but it also does some really stupid things. Depending on which it did most recently for me I change my answer.
The question is does AI understand well enough to maintain that thing for whatever maintenance I need to do in the future?
