Just because humans have additional sources of irrationality doesn't mean they don't also have irrationality based on the same lack of self-awareness that LLMs exhibit.

I could understand that argument as follows: LLMs fill in the gaps in a creative but predictable way. Humans fill in the gaps in creative but unpredictable ways. The creativeness level is affected by the ad hoc state of the brain.

I understand that you relate judgement, ethics and emotions to 'understanding'. I'm not convinced. Emotions might as well be an effect of pattern matching. You hear a (pattern matched) type of song, you feel a certain way.

Conversely, human beings with varying particular experiences can come to the same conclusions, because human cognition can abstract from particulars, while LLMs are, at best, statistical and imagist. No two of us ever experience the same set of triangles, but abstraction allows us to form concepts like "triangularity", which means we can understand what it means to be a triangle per se (something that is not concrete or particular, and therefore cannot be visualized), while an LLM can only proceed based on the concrete and particular data of input triangles and derivations introduced into the model. It can never go "beyond" the surface features of the training model's images, as it were, and where the appearance of having done so occurs, it is not via abstraction, but by way of product of human abstraction. From the LLM's perspective, there is no triangle, only co-occurrence of features, while abstraction goes beyond features, stripping them away to obtain the bare, unimaginable form.