They do suck at solving problems correctly, however if you give them an incorrect solution and ask them to spot mistakes, or just ask for a general method to do a problem, it works out.
However, they might not yet compare to the best of humans. The best SO answers probably represent 0.01% of the answers, which is a high bar. I am certain very amazing teachers and professors exist out there in the world whom LLMs can't beat yet but the average can't compete.