Perhaps, but taken with other works in the area, a better picture does emerge regarding this claim. Substantial works already showing reasoning capabilities in GPT-4, which show that these models do reason very well - near human performance for many causal reasoning tasks. (1)

Additionally, there is a mathematical proof that these systems align with dynamic programming, and therefore can perform algorithmic reasoning. (2,3)

1) https://arxiv.org/abs/2305.00050.pdf

2) https://arxiv.org/pdf/1905.13211.pdf

3) https://arxiv.org/pdf/2203.15544.pdf