mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Software

Reply
 
Thread Tools
Old 2004-01-09, 13:14   #12
Danath
 
Danath's Avatar
 
May 2003
Republic of Moldova

23·5 Posts
Default

GSV3MiaC wrote:
Quote:
That works just dandy every time you make a change to the hours/day, but as I said (way, way, upthread) the problem arises the next time a new exponent is assigned by the server - at that point the new completion date seems to get calculated assuming 24 hours/day (unless you change the hours/day again).
I'm quite sure that the client takes "Hours per day" into account correctly, when calculating completion dates. As I said, the only problem I see here is with "RollingAverage" setting.

GSV3MiaC, from your last post it's understandable how it goes. We know that "RollingAverage" is reset to 1000 when you change the "Hours per day" parameter. So when you change it, the completion date is shown correctly. Over time, "RollingAverage" grows, so when you get another exponent, this setting has a value higher than 1000 - and that's why the completion dates at that moment aren't correct! Now, if you change "Hours per day" again (so that "RollingAverage" will be reset to 1000), the completion dates will be correct again. (Note that this can be deduced from the first sentence of your last post. )

That's why I think that if the "RollingAverage" setting would be left to 1000 all the time (or would not be taken into account when calculating completion dates), the completion dates would be correct, no matter what the "Hors per day" would be set to (either 24 h or less).

BTW, George, why is the "RollingAverage" setting needed? Maybe it should be disabled? And why does it grow so high, for Prime95 gets about the same CPU power over time?
Danath is offline   Reply With Quote
Old 2004-01-09, 22:04   #13
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

32×823 Posts
Default

Quote:
Originally posted by Danath
BTW, George, why is the "RollingAverage" setting needed? Maybe it should be disabled? And why does it grow so high, for Prime95 gets about the same CPU power over time?
This is primarily because some users leave the default setting of 24 hours per day even though they turn the machine off. RollingAverage, if done properly, let's subsequent work assignments be better suited for the machine's actual throughput.
Prime95 is online now   Reply With Quote
Old 2004-01-09, 23:08   #14
GSV3MiaC
 
Jan 2004
Shropshire, UK

24 Posts
Default

Quote:
Originally posted by Danath

GSV3MiaC, from your last post it's understandable how it goes. We know that "RollingAverage" is reset to 1000 when you change the "Hours per day" parameter. So when you change it, the completion date is shown correctly. Over time, "RollingAverage" grows, so when you get another exponent, this setting has a value higher than 1000 - and that's why the completion dates at that moment aren't correct! ? [/B]
Yes, but if that were the case I'd expect the completion date for the =current= exponent to grow/be equally optimistic - never actually seen that happen (although by the time a new exponent is assigned I usually only have ~14 days to go on the old one .. but still, a 2x error on that ought be noticeable).

I'll keep an eye on it next time .. sadly this 12-13 hours/day machine is fairly slow, and already has exponent assigned which will keep it busy through 9/April (on the 13 hours/day basis), so it'll be a while before it gets a new one.

Last fiddled with by GSV3MiaC on 2004-01-09 at 23:09
GSV3MiaC is offline   Reply With Quote
Old 2004-01-10, 18:42   #15
Matthias C. Noc
 
Matthias C. Noc's Avatar
 
Dec 2003

23·3 Posts
Default

Could it be possible that another factor is coming into play here? I noted that from the time a new factor is assigned to you, Prime95 sees that as the moment you start crunching this exponent. So when you load – by default – a new exponent five days before finishing the current one, one gets a 5 day penalty (statistically). When these 5 days not used to crunch this exponent are somehow added to calculate the expected date of finishing, then it would be obviously an error. The stats-page is using these wrong dates to calculate the averages shown there.

While five days might not be much when crunching on an exponent for 3 years, it is a big error-factor when crunching only 21 days on average.

Best,

MCN
Matthias C. Noc is offline   Reply With Quote
Old 2004-01-11, 00:21   #16
markhl
 
Apr 2003
California

22×23 Posts
Default

Here is a concrete example. I am using the Prime95 23.7 client. Test -> Status tells me that my current DC exponent will finish Jan 22 and the next exponent will finish Feb. 10.

I just told the client that I will take a 0 day vacation, to force it to send new completion dates to the server. The dates sent are the same as those in Test -> Status, so they are consistent.

The PC is on 16 hours/day, that is the value I entered into Prime95, and RollingAverage is about 1430.

By experience, I know that these dates are wrong; they assume 24 hours / day. Because it is only on 16 hours / day, it will take 50% longer to do each exponent than Prime95 predicts. The current exponent will finish in 1.5 * 12 = 18 days, and the next one will take 1.5 * 19 = 29 days.

I have observed this behavior for months. It is a definite but minor bug.
markhl is offline   Reply With Quote
Old 2004-01-13, 18:59   #17
GSV3MiaC
 
Jan 2004
Shropshire, UK

24 Posts
Default

My 'rolling average' is now up to 1400, and although the current completion dates look reasonable, I don't see any end in sight to it's rolling increase.

A Question for George/Prime95 .. the 'rolling average' seems to get updated several times a day. I assume the logic (I haven't looked at the code) is somehing like 'the machine is supposed to do 10 interations/second, it did 11, so rolling average increases to 1000*11/10' (or by some part of the delta).

Fine - but if the machine is stated to be on for 12 hours, then =over the course of a day= it only does 5/second (on average). Are you accidentally scaling by 1000*11/5, and failing to add in the 12 hours of 1000*0/5 (when the machine is off .. which, of course, you'd have to do retroactively, since you can't update rolling average when it isn't actually running)?
GSV3MiaC is offline   Reply With Quote
Old 2004-01-13, 19:38   #18
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

1CEF16 Posts
Default

This is the code in UpdateRollingAverage - it should be called only twice a day:

Code:
/* Get the current time and when the iterations started */

        time (&current_time);
        start_time = IniGetInt (LOCALINI_FILE, "RollingStartTime", 0);
        if (start_time == 0 || current_time <= start_time) return;

/* Compute the rolling average where the existing rolling average
/* accounts for 90% of the new rolling average and the current */
/* data point accounts for 10%.  The rolling average */
/* measures this computer's speed as compared to our estimated */
/* speed for this CPU.  A value of 1200 means the this computer */
/* is 20% faster than expected.  A value of 1000 means the this computer */
/* is as fast as expected.   A value of 800 means this computer */
/* is 20% slower than expected. */

/* The old formula used prior to version 19.1 (ra = 0.9 * ra + 0.1 * est) */
/* produced poor results on machines that were not on 24 hours a day.  As */
/* an example the wild fluctuations of 2000, 667, 2000, 667, etc. averaged */
/* out to well more than the correct 1000.  Brian Beesley suggested a new */
/* formula (ra = 1 / (0.9 / ra + 0.1 / est)) which works much better. */

        est = est / (current_time - start_time) * 1000.0;
        if (est > 50000.0) return;      /* A safeguard against bogus data */
        if (est < 0.5 * ROLLING_AVERAGE) est = 0.5 * ROLLING_AVERAGE;
        if (est > 2.0 * ROLLING_AVERAGE) est = 2.0 * ROLLING_AVERAGE;

        ROLLING_AVERAGE = (unsigned int)
                (1.0 / (0.9 / ROLLING_AVERAGE + 0.1 / est) + 0.5);
        if (ROLLING_AVERAGE < 20) ROLLING_AVERAGE = 20;
        if (ROLLING_AVERAGE > 4000) ROLLING_AVERAGE = 4000;
        IniWriteInt (LOCALINI_FILE, "RollingAverage", ROLLING_AVERAGE);
Prime95 is online now   Reply With Quote
Old 2004-01-13, 22:52   #19
GSV3MiaC
 
Jan 2004
Shropshire, UK

208 Posts
Default

Thanks for the code fragment . however a couple of questions ..

1) Where does 'est' come from (prior to the first line)

2) What is the basis for the 'twice a day' .. is it based on time, or on some number of iterations being done (The latter would obviously be affected by today's faster machines).

3) Shouldn't the 'start time' get written to the ini file as well as the updated rolling average (maybe that happens outside of the code fragment).

I still have a suspicion that this is getting calculated twice, or more than twice, per day but only for the 'active period' (does start time maybe get reset when Prime starts in the morning?) and that the 'overnight, off period' never figures in the calculation anywhere.
GSV3MiaC is offline   Reply With Quote
Old 2004-01-13, 23:09   #20
GSV3MiaC
 
Jan 2004
Shropshire, UK

1016 Posts
Default

Late thought - that

if (est < 0.5 * ROLLING_AVERAGE) est = 0.5 * ROLLING_AVERAGE

is a problem anyway - if the machine is on 12, off 12, and you updated twice a day, then worst case the lower est would be 0, and the upper one would indeed be 2x the =real= average.

Depends when the two updates happen of course, but you can be pretty sure that one of them is not going to be halfway through the 'off' period, even if the other one is halfway through 'on'.

Last fiddled with by GSV3MiaC on 2004-01-13 at 23:09
GSV3MiaC is offline   Reply With Quote
Old 2004-01-14, 00:38   #21
GSV3MiaC
 
Jan 2004
Shropshire, UK

1016 Posts
Default

OK, I found and read the code. There is a minor problem, in that the 'twice per day' code uses the input 'CPU Hours' (which may be wrong) to guess how many cycles equate to twice/day, however the main glitch is definitely that 'if est < 0.5 ra' line .. this limits the downside too much, you'd get a better result without any such check.

The '2x' upside allows a ~10% increase in the rolling average (if est = 2 * ra), whereas the 0.5x limit on the downside means that ra can only go down by 5% (90% of what is was, + 10% of 50% of what it was) .. should be equally able to float down by 10%.
GSV3MiaC is offline   Reply With Quote
Old 2004-01-14, 14:02   #22
GSV3MiaC
 
Jan 2004
Shropshire, UK

24 Posts
Default

Dear George ..

I've been doing some testing, and I think there are actually two problems.

1) You shouldn't be limiting 'est' to 0.5*RA, =nor= to 2*RA .. actually the first limit is sort of controlled by the maths, but the second one is definitely wrong .. i.e. a machine running 1 hour/day, and turning in two RA adjustments, would come in at
(24/47)*1000 for the 'first reading' and (24/1)*1000 for the second one (i.e. taking half an hour to do half a day's work).

Having two RA adjustments per day is probably a bad thing anyway, when you are trying to estimate 'hours per day' (among other things), where an 'in-day' sample is bound to give a daft answer.

2) There is a bug, which I can't find, which causes 'rolling start time' to be reset to 0 whenever I restart the machine (or even stop and restart prime95), IF

a) the machine is working on LL testing (and is >50% of the way through?) AND

b) there is a second work unit in the queue (which requires factoring?)

The effect of this is that the first RA update each day gets dumped (since it discovers that rolling start time was 0). The second (and possibly third) one happen just fine. The first one, of course, if the one that contains the 'overnight' (off time) data.

HTH
GSV3MiaC is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
64-bit lnx vs win client, Prime95.exe under Wine? xorbe Software 13 2009-03-18 07:22
split a prime95 queue & client installation joblack Information & Answers 1 2009-01-06 08:45
Prime95 client stuck? edorajh Software 7 2003-11-08 17:38
client problem cameloid Software 1 2002-10-16 14:30
Linux mprime client v22.8 problem Prime Monster Software 6 2002-08-29 11:14

All times are UTC. The time now is 00:49.

Fri Apr 16 00:49:50 UTC 2021 up 7 days, 19:30, 0 users, load averages: 1.63, 1.64, 1.65

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.