And it's this expansion aspect that seems crucial to me.
Endless rehashing of inputs can lead to seemingly creative outcomes, but that doesn't mean any actual expansion occurs. New works are created/discovered, but are new underlying principles discovered?
I recall seeing a Two Minute Papers episode about an AI built to be curious, or specifically - explore routes in a maze where it couldn't predict the outcome.
The easiest way to block it was to present an animated wall. The AI would just stare at it and not go anywhere.
The generation side is actually probably easier than the critic side. I don't know of any network that is capable of determining if something is aesthetically pleasing in an abstract sense, instead they always compare to known examples.
Humans do the same - recombine the same limited vocabulary in new ways to address new goals. We rarely invent original meanings that could not have been expressed before.
