Even if LLMs were "a memorization machine" (they're bad at that), the statement is obviously false because actutal literal memorisation machines (books, video recorders, Google) cannot pass these exams.

LLMs only started to because they could follow the questions.

But even that aside, it doesn't matter why LLMs can do what they can do or what else can also do that, what matters is that it would take most humans several years to get to the level of current LLMs in a subject that human isn't already familiar with.