The main point of HT is to reduce the cost of context switching by keeping twice the number of contexts close to the core. I would guess that parts of the process context like program counter, TLB, etc live inside the 'HT' and would have to be saved/restored every time the process moves between threads, even on the same core. Reserving both 'HT' on a core gets you cache locality, but isn't there a cost to moving the process back and forth, even if that data is in L1/L2?

(I'm looking at 'lstopo' from package 'hwloc', Linux on my Haswell Xeon: 10MB shared L3, 256KB L2, 32KB L1{d,i} per core)

Given my (educated) guess, I've told irqbalance to put interrupts only on 'thread 0' and then I schedule cpu-intensive tasks to 'thread 1' and schedule them very-not-nicely. Linux seems pretty good about keeping everything else on 'thread 0' when I have 'thread 1' busy so I don't do any further management.

I can have 4 cores 'thread 1' pegged at 100% with no impact on interactive or I/O performance.