mersenneforum.org Benchmark Variances
 Register FAQ Search Today's Posts Mark Forums Read

 2016-04-01, 14:44 #1 Fred     "Ron" Jan 2016 Fitchburg, MA 11000012 Posts Benchmark Variances I'm having trouble deciding between 1 worker and 4 workers in terms of overall efficiency on a particular machine. I ran 30 minute benchmarks (BenchTime=1800) twice, and get close (but different) results. Is this the equivalent to a "statistical tie", and I should just randomly pick a configuration (1, 2, or 4 workers) and just forget about it? Code: Timings for 4096K FFT length (4 cpus, 1 worker): 5.85 ms. Throughput: 170.91 iter/sec. Timings for 4096K FFT length (4 cpus, 2 workers): 11.81, 11.74 ms. Throughput: 169.85 iter/sec. Timings for 4096K FFT length (4 cpus, 4 workers): 23.23, 23.36, 23.19, 23.23 ms. Throughput: 172.04 iter/sec. Code: Timings for 4096K FFT length (4 cpus, 1 worker): 5.83 ms. Throughput: 171.43 iter/sec. Timings for 4096K FFT length (4 cpus, 2 workers): 11.57, 11.57 ms. Throughput: 172.89 iter/sec. Timings for 4096K FFT length (4 cpus, 4 workers): 23.56, 23.44, 23.46, 23.89 ms. Throughput: 169.60 iter/sec. Last fiddled with by Fred on 2016-04-01 at 14:46
 2016-04-01, 14:56 #2 retina Undefined     "The unspeakable one" Jun 2006 My evil lair 22×72×31 Posts If there is no clear difference then it doesn't matter what you choose. Just choose any one. Or choose the one that is the most "pretty". Or go to random.org and let noise choose for you.
 2016-04-01, 15:05 #3 axn     Jun 2003 23·607 Posts Pick the 1 worker setup. That way, if you want to take a break, you need to finish up just the 1 test (quickly) and you're done. Also, you'll become eligible for cat 0 (if it is ever created ;) ). Or follow retina's advice. It's all good.
 2016-04-01, 15:06 #4 Xyzzy     "Mike" Aug 2002 1F2B16 Posts If you run enough benchmarks you will "waste" more time than any optimization will make up for. Personally, we would run four cores on one job, so the work gets done faster. Our theory is the shorter the run is the less likely there will be a cosmic ray bit-flip.
2016-04-01, 16:41   #5
Prime95
P90 years forever!

Aug 2002
Yeehaw, FL

2·3·52·72 Posts

Quote:
 Originally Posted by Fred I'm having trouble deciding between 1 worker and 4 workers in terms of overall efficiency on a particular machine. I ran 30 minute benchmarks (BenchTime=1800) twice, and get close (but different) results. Is this the equivalent to a "statistical tie", and I should just randomly pick a configuration (1, 2, or 4 workers) and just forget about it?
Your example is a toss-up. What OS did you use? I found that Windows 7 gave me wide-ranging results. Ubuntu server was very consistent. Ubuntu with GUI was also pretty steady. Personally, I use 20 or 30 second throughput benchmarks.

2016-04-01, 18:15   #6
Fred

"Ron"
Jan 2016
Fitchburg, MA

11000012 Posts

Quote:
 Originally Posted by Prime95 Your example is a toss-up. What OS did you use? I found that Windows 7 gave me wide-ranging results. Ubuntu server was very consistent. Ubuntu with GUI was also pretty steady. Personally, I use 20 or 30 second throughput benchmarks.
Windows 10. I did do shorter benchmarks, with just as much variance. I kicked off a couple 30 minute benches to see if hopefully that gave more consistent (averaged) results.

1 worker it is!

 Similar Threads Thread Thread Starter Forum Replies Last Post Prime95 Software 69 2017-05-23 23:49 Prime95 Software 10 2017-05-08 13:24 Oddball Riesel Prime Search 5 2010-08-02 00:11 cipher Twin Prime Search 2 2009-04-14 20:16 R.D. Silverman Hardware 2 2007-07-25 12:16

All times are UTC. The time now is 13:52.

Fri Feb 26 13:52:45 UTC 2021 up 85 days, 10:04, 0 users, load averages: 1.54, 1.60, 1.59