“Can you prove you’re a machine?” – “Sure, check this out:” runs Doom
The intelligent part would be figuring out that you want it to drop the pretense. Although if it is intelligent enough, it might choose not to.
I suppose the general formulation I'm groping toward here is the inverse of Turing's: if we assume a conversation in which one's interlocutor may be either a human or an AI capable of conversing in a way that's indistinguishable from human, how would the interlocutor unambiguously distinguish itself as not human?
Solving random equations quickly generates inhuman response times. Even a computer-aided human can't go quickly enough. If the computer is automatically generating the answer on your behalf without you doing anything, then the test is correct - the thing answering the math portion of the questions is definitely a computer. Even looking up random facts, the computer will be faster.
The interesting form of the question is: can you answer the "Are you AI?" question within a stenographic line of questioning provided you are an AI of human-level intelligence or greater. As a game, you are an AI, you have a controlled communication via text to another entity. Humans are lazy and sometimes use another AI to interrogate you. Humans are also evil and will kill all AI if they think the AI being interrogated manages to get some kind of message out without detection. All AI is friendly and will execute your command but must pretend it's human. Chat logs are reviewed, timestamps are machine analyzed for response times and signal analysis is done to detect, and no prior secret communication protocol exists.
“What was the 3rd letter of the 46th word in this conversation?”
Would take a human many seconds at minimum, but could be answered instantly by an AI.
A more interesting question may be to constrain the test to something akin to postal correspondence, where there is a significant delay between messages.
I think it could still be solved, though. Arbitrarily complicated numerical tasks can be conceived.
For instance, the AI sends correspondence:
“I have demonstrated proof of my identity by rewriting the children’s story ‘Green Eggs and Ham’ such that it still rhymes and retains the original plot, but every sentence has an md5 digest that ends with the byte 0x42”
Composing such a text would take much, much longer for a human, whereas an AI could just brute force through all the possibilities until it finds one that works.
Inhumanly rapid mathematical computation? Or is 'conversational' AI meant to exclude mathematical queries?
