That's probably part of it. Radiation is certainly a bit harder to detect with our human senses. Maybe if we put radiation detectors in every home and printed out XKCD dose effect charts people could make more risk-informed (as opposed to fear/uncertainty-informed) decisions.
https://xkcd.com/radiation/
Keep in mind that normal fossil fueled power plants are killing about 4.2 million people per year via air pollution, which is also largely invisible. I don't quite understand why radiation killing up to 4000 freaks people out so much more vs. air pollution killing 4.2 million/year.
https://www.who.int/airpollution/en/