Intel's doc [1] if SMT is not disabled, not only does the OS need to perform group scheduling (only allow threads that trust each other to run on the same core at the same time) but the OS also has to interrupt the sibling thread running on the same core whenever user mode code enters kernel mode, synchronizing syscalls. For compute-heavy applications, this might be ok, but for servers doing a lot of syscalls this might be terrible.
I haven’t seen a Linux implementation of this yet, but I suspect the performance is beyond terrible. Intel’s doc says:
> There is no hardware support for atomic transition of both threads between kernel and user states, so the OS should use standard software techniques to synchronize the threads.
Ha, “standard software techniques”. Let’s see: suppose one thread finishes a syscall. Now it sets some flag telling the other core to wake up and go back to user code as well. But the other core is running with interrupts on. So now we sit there spinning until the other core is done, and then both cores turn off interrupts and recheck that they both still want to go to user mode. This involves some fun lock-free programming. I suppose one could call this a “standard software technique” with a somewhat straight face.
Meanwhile, the IPI on entry involves an IPI and the associated (huge!) overhead of an x86 interrupt. x86 interrupts are incredibly slow. The IPI request itself is done via the APIC, and legacy APIC access is also incredibly slow. There’s a fancy thing called X2APIC that makes it reasonably fast, but for whatever reason, most BIOSes don’t seem to enable it.
I suspect that relatively few workloads will benefit from all of this. Just turning off SMT sounds like a better idea.
1 - See "Synchronized ring 0 entry and exit using IPIs" in https://software.intel.com/security-software-guidance/insigh...