Okay maybe it is indeed very predictable what would happen on 4chan, but I hope you get my point.
Here the AI is just a tool, we shouldn't categorize them as some "special" software.
Not particularly the best comparison, we are talking here about applications and services which we sell and we have to make sure that we have the confidence that we can trust our own systems and the customer can rely on our word and engineering skills. Right now we are still in the primordial soup of AI. As long as we do not have proper methods to verify and certify our models, they are dangerous and unpredictable.
This is not a new issue but is heavily researched in the certifiable AI area. If you just turning million knobs randomly based on an architecture and the biased data you have, of course you don't really known what you are doing.
It can obviously have some effect on the so-called ‘real world’, or people wouldn’t pour money into developing & using it. It would be a unique feat to be powerful with no chance of any of the effects being harmful.
From there, it’s just arithmetics: what harms and benefits do we expect with what probabilities, and how does this change depending on the distribution scheme?