mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Software

Reply
 
Thread Tools
Old 2016-04-01, 14:44   #1
Fred
 
Fred's Avatar
 
"Ron"
Jan 2016
Fitchburg, MA

97 Posts
Default Benchmark Variances

I'm having trouble deciding between 1 worker and 4 workers in terms of overall efficiency on a particular machine. I ran 30 minute benchmarks (BenchTime=1800) twice, and get close (but different) results. Is this the equivalent to a "statistical tie", and I should just randomly pick a configuration (1, 2, or 4 workers) and just forget about it?

Code:
Timings for 4096K FFT length (4 cpus, 1 worker):  5.85 ms.  Throughput: 170.91 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 11.81, 11.74 ms.  Throughput: 169.85 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 23.23, 23.36, 23.19, 23.23 ms.  Throughput: 172.04 iter/sec.
Code:
Timings for 4096K FFT length (4 cpus, 1 worker):  5.83 ms.  Throughput: 171.43 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 11.57, 11.57 ms.  Throughput: 172.89 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 23.56, 23.44, 23.46, 23.89 ms.  Throughput: 169.60 iter/sec.

Last fiddled with by Fred on 2016-04-01 at 14:46
Fred is offline   Reply With Quote
Old 2016-04-01, 14:56   #2
retina
Undefined
 
retina's Avatar
 
"The unspeakable one"
Jun 2006
My evil lair

5,879 Posts
Default

If there is no clear difference then it doesn't matter what you choose. Just choose any one. Or choose the one that is the most "pretty". Or go to random.org and let noise choose for you.
retina is offline   Reply With Quote
Old 2016-04-01, 15:05   #3
axn
 
axn's Avatar
 
Jun 2003

12AC16 Posts
Default

Pick the 1 worker setup. That way, if you want to take a break, you need to finish up just the 1 test (quickly) and you're done. Also, you'll become eligible for cat 0 (if it is ever created ;) ).

Or follow retina's advice. It's all good.
axn is offline   Reply With Quote
Old 2016-04-01, 15:06   #4
Xyzzy
 
Xyzzy's Avatar
 
"Mike"
Aug 2002

3·19·137 Posts
Default

If you run enough benchmarks you will "waste" more time than any optimization will make up for.

Personally, we would run four cores on one job, so the work gets done faster. Our theory is the shorter the run is the less likely there will be a cosmic ray bit-flip.

Xyzzy is offline   Reply With Quote
Old 2016-04-01, 16:41   #5
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

2×3×1,193 Posts
Default

Quote:
Originally Posted by Fred View Post
I'm having trouble deciding between 1 worker and 4 workers in terms of overall efficiency on a particular machine. I ran 30 minute benchmarks (BenchTime=1800) twice, and get close (but different) results. Is this the equivalent to a "statistical tie", and I should just randomly pick a configuration (1, 2, or 4 workers) and just forget about it?
Your example is a toss-up. What OS did you use? I found that Windows 7 gave me wide-ranging results. Ubuntu server was very consistent. Ubuntu with GUI was also pretty steady. Personally, I use 20 or 30 second throughput benchmarks.
Prime95 is offline   Reply With Quote
Old 2016-04-01, 18:15   #6
Fred
 
Fred's Avatar
 
"Ron"
Jan 2016
Fitchburg, MA

97 Posts
Default

Quote:
Originally Posted by Prime95 View Post
Your example is a toss-up. What OS did you use? I found that Windows 7 gave me wide-ranging results. Ubuntu server was very consistent. Ubuntu with GUI was also pretty steady. Personally, I use 20 or 30 second throughput benchmarks.
Windows 10. I did do shorter benchmarks, with just as much variance. I kicked off a couple 30 minute benches to see if hopefully that gave more consistent (averaged) results.

1 worker it is!
Fred is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
29.2 benchmark help Prime95 Software 69 2017-05-23 23:49
29.2 benchmark help #2 (Ryzen only) Prime95 Software 10 2017-05-08 13:24
LLR benchmark thread Oddball Riesel Prime Search 5 2010-08-02 00:11
Does anyone have i7 920? for Benchmark? cipher Twin Prime Search 2 2009-04-14 20:16
Benchmark Weirdness R.D. Silverman Hardware 2 2007-07-25 12:16

All times are UTC. The time now is 20:39.

Fri Nov 27 20:39:50 UTC 2020 up 78 days, 17:50, 3 users, load averages: 2.00, 1.65, 1.63

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.