When you lower the friction of an action sufficiently, it causes a qualitative change in the emergent behavior of the whole system. It's like how a little damping means the difference between a bridge you can safely drive over versus a galloping Gertie that resonates until it collapses.
When a human has to choose and put some effort into regurgitating a piece of information, there is a natural decay factor in the system where people will sometimes not bother to repeat something if it doesn't seem valuable enough to them. Sure, things like urban legends and old wive's tales exploit bugs in our information prioritization. But, overall, it has the effect of slowly winnowing out nonsense, misinformation, and other low value stuff. Meanwhile, information that continues to be useful continues to be worth the effort of repeating.
Compared to the print and in-person worlds before, things got much worse just with social media where a human was still in the loop but the effort to rebroadcast was nil. This is exactly why we saw a massive rise in misinformation in the past couple of decades.
With ChatGPT and humans completely out of the loop, we will turn our information systems into galloping Gertie and they will resonate with nonsense and lies until the whole system falls apart.
We are witnessing the first cracks now. Look at George Santos, a candidate who absolutely should have never won a single election but managed to because information pipelines about candidates are so polluted with junk and nonsense that voters didn't even realize he was a con man. Not even a sophisticated one, just a huckster able to hide within the sea of information noise.
