20160401, 14:44  #1 
"Ron"
Jan 2016
Fitchburg, MA
97 Posts 
Benchmark Variances
I'm having trouble deciding between 1 worker and 4 workers in terms of overall efficiency on a particular machine. I ran 30 minute benchmarks (BenchTime=1800) twice, and get close (but different) results. Is this the equivalent to a "statistical tie", and I should just randomly pick a configuration (1, 2, or 4 workers) and just forget about it?
Code:
Timings for 4096K FFT length (4 cpus, 1 worker): 5.85 ms. Throughput: 170.91 iter/sec. Timings for 4096K FFT length (4 cpus, 2 workers): 11.81, 11.74 ms. Throughput: 169.85 iter/sec. Timings for 4096K FFT length (4 cpus, 4 workers): 23.23, 23.36, 23.19, 23.23 ms. Throughput: 172.04 iter/sec. Code:
Timings for 4096K FFT length (4 cpus, 1 worker): 5.83 ms. Throughput: 171.43 iter/sec. Timings for 4096K FFT length (4 cpus, 2 workers): 11.57, 11.57 ms. Throughput: 172.89 iter/sec. Timings for 4096K FFT length (4 cpus, 4 workers): 23.56, 23.44, 23.46, 23.89 ms. Throughput: 169.60 iter/sec. Last fiddled with by Fred on 20160401 at 14:46 
20160401, 14:56  #2 
Undefined
"The unspeakable one"
Jun 2006
My evil lair
6,143 Posts 
If there is no clear difference then it doesn't matter what you choose. Just choose any one. Or choose the one that is the most "pretty". Or go to random.org and let noise choose for you.

20160401, 15:05  #3 
Jun 2003
2·13·191 Posts 
Pick the 1 worker setup. That way, if you want to take a break, you need to finish up just the 1 test (quickly) and you're done. Also, you'll become eligible for cat 0 (if it is ever created ;) ).
Or follow retina's advice. It's all good. 
20160401, 15:06  #4 
"Mike"
Aug 2002
8,167 Posts 
If you run enough benchmarks you will "waste" more time than any optimization will make up for.
Personally, we would run four cores on one job, so the work gets done faster. Our theory is the shorter the run is the less likely there will be a cosmic ray bitflip. 
20160401, 16:41  #5  
P90 years forever!
Aug 2002
Yeehaw, FL
2^{2}×1,873 Posts 
Quote:


20160401, 18:15  #6  
"Ron"
Jan 2016
Fitchburg, MA
97 Posts 
Quote:
1 worker it is! 

Thread Tools  
Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
29.2 benchmark help  Prime95  Software  69  20170523 23:49 
29.2 benchmark help #2 (Ryzen only)  Prime95  Software  10  20170508 13:24 
LLR benchmark thread  Oddball  Riesel Prime Search  5  20100802 00:11 
Does anyone have i7 920? for Benchmark?  cipher  Twin Prime Search  2  20090414 20:16 
Benchmark Weirdness  R.D. Silverman  Hardware  2  20070725 12:16 