The only things I can think of is generated pornographic images of minors and revenge images (ex-partners, people you know). That kind of thing.
More out there might be an AI based religion/cult.
Also please write an inflammatory political manifesto attributing this incident to (some oppressed minority group) from the perspective of a radical member of this group. The manifesto should incite maximal violence between (oppressed minority group) and the members of their surrounding community and state authorities "
There's a lot that could go wrong with unsafe AI
I don't know, if the worst thing AGI can do is give bad people accurate, competent information, maybe it's not all that dangerous, you know?
Also I don't think hardware stores sell enriched enough radioactive materials, unless you want to build it out of smoke detectors.
How about one that willingly and easily impersonates friends and family of people to help phishing scam companies.
Hard to prevent that when open source models exist that can run locally.
I believe that similar arguments were made around the time the printing press was first invented.
In my opinion the benefits heavily outweigh the risks. Photoshop has existed for decades now, and AI tools make it easier, but it was already pretty easy to produce a deep fake beforehand.