Good question.

I just asked text-davinci-002 to complete this prompt:

    Here is some python code:

    def reverse(s): 
        return s[::-1]

    What would this function return for the input "hello"?

And it actually responds:

    The function would return "olleh".

Sadly, when I asked it to try "encyclopedia", it said:

    "aidnocypilecne"

Which is a brave attempt, but not quite right.

To see if it was just making assumptions based on the function name, I tried to trick it with

    def reverse(s): 
        return s + 2

    What would this function return for the input "hello"?

But GPT-3 is too clever for that:

    It would return "hello2".

Also, it even recognizes what the function will actually do, when it's named misleadingly:

    def getLength(s): 
         return s[::-1]

    What would this function return for the input "hello"?

Still produces:

    It would return "olleh".