Well the least they could do imo would be to post or comment that a question is a duplicate of another or link to the top voted answer. Similar to how users on HN post links to "on going discussion threads" for duped posts. Its grunt work that these bots should at least be able to regurgitate or find easily.

Also theres a chance these LLMs have access to other tech forums in addition to stack overflow and could possibly provide a solution. For example GitHub has actually been the better source for me when debugging issues. Usually you can go to the repo and search the issues and read comments with solutions or workarounds.

But aside from that i am in agreeance with you that these bots will struggle to provide new, non regurgitated answers and could potentially cause more harm than good