Hyperthreading broken in Skylake and Kaby Lake?

[GP2]

The source of this information is this post in the Debian mailing list.

There are reddit threads in r/linux and in r/hardware and r/intel and r/programming.

Anyone heard about this?

PS,
The Skylake processors on Google Compute Engine are model 85, stepping 3. I have turned HyperthreadLL=1 on for mprime and ran about a dozen or so successful double checks, and am currently running several instances of Mlucas with hyperthreading (using -cpu 0:1 with a single-core virtual machine) but they haven't run to completion yet.

For the cloud, I'd probably keep hyperthreading on despite the risk, because there are no consequences to data corruption in a virtual machine other than a bad LL result, and on GCE (unlike Amazon AWS) the benchmarks are considerably faster with hyperthreading enabled.

[ewmayer]

Originally Posted by GP2

For the cloud, I'd probably keep hyperthreading on despite the risk, because there are no consequences to data corruption in a virtual machine other than a bad LL result, and on GCE (unlike Amazon AWS) the benchmarks are considerably faster with hyperthreading enabled.

No consequences - other than wasted cycles and $, you mean? (I realize you are likely still using your free-trial-GCE-$ for your testing there, but 'industrial strength' GCE users will be unlikely to share your 'so what' attitude. :) Since it is important to get a sense of frequency of occurrence of the HT bugs, may I presume you are running DCs on GCE?

[GP2]

Originally Posted by ewmayer

No consequences - other than wasted cycles and $, you mean? (I realize you are likely still using your free-trial-GCE-$ for your testing there, but 'industrial strength' GCE users will be unlikely to share your 'so what' attitude. :) Since it is important to get a sense of frequency of occurrence of the HT bugs, may I presume you are running DCs on GCE?

It already did a dozen or so correct DCs using mprime, and from what I gather from reading the reddit threads, the bug is only triggered under rare circumstances. So I am leaving hyperthreading on for Mlucas and we'll see what happens. On GCE (unlike AWS) there would be a major performance impact to turning it off.

What I meant was, with a virtual machine you don't have to worry about data corruption somehow affecting the operating system files, since you can just create a new machine in seconds.

[ewmayer]

Originally Posted by GP2

It already did a dozen or so correct DCs using mprime, and from what I gather from reading the reddit threads, the bug is only triggered under rare circumstances. So I am leaving hyperthreading on for Mlucas and we'll see what happens. On GCE (unlike AWS) there would be a major performance impact to turning it off.

What I meant was, with a virtual machine you don't have to worry about data corruption somehow affecting the operating system files, since you can just create a new machine in seconds.

Were you using single-threaded mode for your mprime runs? IIRC you got best performance for mprime 1-threaded, vs mlucas 2-threaded. I'm wondering whether having HT enabled but running 1-threaded is less likely to hit the bug than running 2-threaded, i.e. 1 thread for each of the 2 logical cores enabled by HT. In any event, your Mlucas DC results, once they complete, will tell us more.

[GP2]

Originally Posted by ewmayer

Were you using single-threaded mode for your mprime runs? IIRC you got best performance for mprime 1-threaded, vs mlucas 2-threaded. I'm wondering whether having HT enabled but running 1-threaded is less likely to hit the bug than running 2-threaded, i.e. 1 thread for each of the 2 logical cores enabled by HT. In any event, your Mlucas DC results, once they complete, will tell us more.

I used HyperthreadLL=1 for the mprime double-checks.

I recall finding that enabling it made mprime run faster, although it's hard to quantify exactly how much. There is considerably more variability in the benchmarks when running on GCE compared to AWS, I"m not sure why. GCE has the interesting property that they can migrate a running process from one server to another without stopping it, and there might be some heterogeneity in terms of Skylake processor speeds and models for all I know.