I think as well as the obvious connection to nuclear warfare, part of the public fear of nuclear technology comes directly from how highly radioactive objects mimic the idea of a "curse".
In reality there's many forms of industrial pollution that are arguably scarier than nuclear waste, but none are quite so eerie in how their harm is caused. Even though it represented the very worst practices for these materials, the idea of a place like Lake Karachay where even half an hour on its shore would have killed you without you even knowing you were doomed is really unsettling. I don't think nuclear technology should be avoided, but there's definitely a formidable image problem here I think.