At least for mathematics I have hardly seen any "nutty mathematical theories" (I am a PhD student in mathematics, but not a physicist, so I cannot say anything about physics here). The reason that I see is that for a math text to be considered as solid, it has to contain good, understandable proofs - which are hard to write by "crackpots". To be more precise: On sceptic's blogs I have of course seen links to papers containing "mathematical crackpottery", but I have hardly ever seen those "in the open countryside".

> But then why do you judge the conspiracy theories to be nutty, if they are on just as firm an epistemological footing as actual history?

But to elaborate on your argument: Because of the dubious epistemological footing indeed "actual history" has not the highest reputation to me. The reason I disregard lots of "nutty conspiracy theories" rather lies in the fact that many "conspiracists" have a tendency to find conspiracies in other topics, too, where falsification is much better possible. Thus I tend to judge by looking at the track record of the respective person.