An LLM is not type 0. It always finishes in finite time so it is not Turing complete.

I asked Copilot

   answer yes or no,  is the following Java method syntactically well formed
   
   static int aMethod() { return "5"; }

and got what I thought was the wrong answer

   No.
   It’s syntactically valid as Java code, but it will not compile because it returns a String where 
   an int is required.

because I hadn't specified clearly that I was talking about the type 2 CFG of the parser as opposed to the type 1 behavior of the compiler as a whole. [1] I had a good conversation with copilot about it and I'm sure I'd get better results with a better prompt... It would make a good arXiv paper to pose an LLM grammatical recognition problems by prompting

   here is a set of rules: ...  is the production ... in the grammar?

with a wide range of cases. Somebody who just tries a few examples might be impressed by its capability but if you were rigorous about it you would conclude that an LLM pretends to be able to recognize grammars but can't actually do it.

And that's true about everything they do, one could argue that "in an exact sense, LLM can't do anything at all") They'll make a try at a weird question like

   what percent of North Americans would recognize the kitsune hand gesture?

which is a public opinion research question similar in character to

   what is lowest mass eigenstate of the neutrino?

in that it could be answered rigorously (but still in terms of probability, even hep results have p-values)

[1] javac implements type 1 behavior in the java language which is a substrate