It’s actually weirdly similar to the kind of tricks people use for mental feats like memorizing the order of a complete deck of cards or repeating back a long list of words in reverse order.
When you think about every mental task GPT3 is being asked to do as being something it is being asked to perform immediately and without having prepared and as fast as possible this makes a lot more sense.
Like, a reasonable human response to “quick! What’s encyclopedia backwards?!” Would be more like
“Er.. right. A. I. D. E. O? Oh wait is it one of those OE ligature things? P. A. No, O. P. Hang on did I already say P?”