The thing is, you can interact with this new kind of actor as much as you need to to judge this -- make up new problems, ask your own questions. "LLMs can't think" has needed ever-escalating standards for "real" thinking over the last few years.
Gary Marcus made a real-money bet about this.