You're right, we do know how it works. Your mistake is concluding that because we know how LLMs work and they're not that complicated, but we don't know how the brain works and it seems pretty complicated, therefore the brain can't be doing what LLMs do. That just doesn't follow.

You made exactly the same argument in the opposite direction, asking if my rubric for "can reason with intelligence and have an opinion" is "seems like it", and your rubric for "thinking is not a token predictor driven by matrix multiplications" is "seems like it".

You can make a case for the plausibility of each conclusion, but that's doesn't make it a fact, which is how you're presenting it.