It is easy to blame some entity everyone knows exits, but at the same time, that entity is more like a group which doesn't fight back against it. Even if they fight back and criticize the government here and there, the latter own the media and that becomes a non-story.
Right wing governments always need a boogeyman to rally against. Today it is EU, LGBT+ groups, Roma etc. Tomorrow it might be you and me.