In fairness, people do the same thing with "far right". I've met some people who would likely characterize President Obama as far right.
This particular pathology seems to arise from the idea that "far $TYPE" is inherently de-legitmizing. And once you've de-legitimized someone or some position, you don't actually have to take them seriously, so...