If Kernel Lockdown is enabled, a zero-day exploit is required to bypass module restrictions without a reboot.
Unfortunately, threat actors tend to have a stash of them and the initial entry vector often involves one (container or browser sandbox escape), and once you have that, you are in ring 0 already and one flipped bit away from loading the module.
The Linux kernel is not really an effective privilege boundary.