Benchmark and Reputation calculation
 

Hello,

please have a look at this part of my CPU's benchmark (Intel [EMAIL="E6600@2.4GHz"]E6600 @ 2.4GHz[/EMAIL], Dualcore)

Best time for 58 bit trial factors: 4.509 ms.
Best time for 59 bit trial factors: 4.506 ms.
Best time for 60 bit trial factors: 4.463 ms.
Best time for 61 bit trial factors: 4.441 ms.
Best time for 62 bit trial factors: 7.450 ms.
Best time for 63 bit trial factors: 7.435 ms.
Best time for 64 bit trial factors: 7.199 ms.
Best time for 65 bit trial factors: 7.163 ms.
Best time for 66 bit trial factors: 7.158 ms.
Best time for 67 bit trial factors: 7.126 ms.

Now I made some tf's (on a M-number with exponent ~ 3.2e6) with this times for the screen outputs (every 10,000 Iterations):

TF from 2^60 to 2^61: screen output every 45 sec
TF from 2^61 to 2^62: screen output every 44 sec
TF from 2^62 to 2^63: screen output every 74 sec
TF from 2^63 to 2^64: screen output every 74 sec
TF from 2^64 to 2^65: screen output every 73 sec

Obviously the time for 62 bit trial factors is still as fast as the tf with 61 bit trial factors. and the second 'jump' in the list (from ~7.45ms to ~7.15ms) seems to be between 64 bit factors and 65 bits factors (which make sense with a 32 bit processor).

:redface: I can use this to get a lot more of 'reputation GHhzdays' with tf:

Doing 1 hour with 61 bit tf I get ~ 2.9 GHzhour rep.
Doing 1 hour with 62 bit tf I get ~ 3.6 Ghzhour rep. (!)
Doing 1 hour with 63 bit tf I get ~ 2.8 GHzhour rep.
Doing 1 hour with 64 bit tf I get ~ 3.0 GHzhour rep.
Doing 1 hour with 65 bit tf I get ~ 2.8 GHzhour rep.

I think the mount of reputation I get while doing 62 bit tf is, because the proggy already takes the high 7.45ms per division to calculate the reputation, but I need only 4.44ms in reality to do the division.
This liitle error in the offset of the benchmark could be the reason for wrong calculations of the durations of trial factorings, too. If I get a tf which needs 1 hour on my computer, I get a calculation of ~2 hours in the status window. If I request tf-asignments for 3 days I only get work for ~ 1.5 days (and the status windows estimates ~ 3 days...).

Btw: a re-calculation of some work I did a while ago gave:

Doing 1 hour of LLT I get ~ 2.35 GHzhour rep.
Doing 1 hour P-1 factoring I get ~ 1.85 GHzhour rep.
(no experience for ECM atm)

The rep. for the LLT confirms with my CPU data (one core of my CPU should make ~ 2.4 GHzhour/physically hour)
The rep. for the P-1 factoring seems to be a little bit low, but maybe this is wished by the project or my computer is bad for doing P-1 (only 2 GB of RAM).
But I'm interested in the weighting of the rep. calculation for the different kinds of work.
IMHO the LLT should give the most of rep/hour and factorings less rep./time. Are there any information available how this weighting is wanted by the project?

thanks in advance (and forgive my english please :cry:)

Matthias

I don't think "reputation" is the appropriate word in this context. Unless of course you're near the top of the factoring rankings. :wink:

i think MatWur intends reputation to mean credit

[QUOTE=MatWur-S530113;149961]
Best time for 60 bit trial factors: 4.463 ms.
Best time for 61 bit trial factors: 4.441 ms.
Best time for 62 bit trial factors: 7.450 ms.
Best time for 63 bit trial factors: 7.435 ms.
Best time for 64 bit trial factors: 7.199 ms.

Now I made some tf's (on a M-number with exponent ~ 3.2e6) with this times for the screen outputs (every 10,000 Iterations):

TF from 2^60 to 2^61: screen output every 45 sec
TF from 2^61 to 2^62: screen output every 44 sec
TF from 2^62 to 2^63: screen output every 74 sec
TF from 2^63 to 2^64: screen output every 74 sec
TF from 2^64 to 2^65: screen output every 73 sec

Doing 1 hour with 61 bit tf I get ~ 2.9 GHzhour rep.
Doing 1 hour with 62 bit tf I get ~ 3.6 Ghzhour rep. (!)
Doing 1 hour with 63 bit tf I get ~ 2.8 GHzhour rep.
Doing 1 hour with 64 bit tf I get ~ 3.0 GHzhour rep.
Doing 1 hour with 65 bit tf I get ~ 2.8 GHzhour rep.

[/QUOTE]

If I understand your question regarding rep (credit) for TF ... then as I interpret it (and as I have trimmed your post) "62 bit trial factors" corresponds to "TF from 2^62 to 2^63". I believe interpreting it this way should eliminate the 3.6 Ghz blip on your last chart above.

[QUOTE=MatWur-S530113;149961]But I'm interested in the weighting of the rep. calculation for the different kinds of work.
IMHO the LLT should give the most of rep/hour and factorings less rep./time. Are there any information available how this weighting is wanted by the project?[/QUOTE]I strongly disagree : the factoring limits are calculated for biggest overal throughput of the project, they are based on benchmarking. This means that for the project TF and P-1 are as valuable as LL testing. Do not forget that every factor found in a day of TF or a day and a half of P-1 means that TWO LL tests of a month each do not have to be done.

The current credit retribution scheme tries to be as fair as possible. Of course the actual weight (the amount of credit one can earn on a given CPU per time unit) depends on the CPU and its interaction with the rest of a workstation (other CPU's or cores, memory...) The previous credit system was heavely balanced towards successful TF and LL ; unsuccessful TF earned little credit and unsuccessful P-1 factoring earned only a symbolic amount of credit. The conclusion would have been, for somebody looking for credit, to do LL only.

Jacob

Hello again,

first thank you at all for the answers. Of course I meant 'cpu-credit' instead of 'reputation', this happens if one write a post at 4:00 in the morning:redface:

petrw1 said exactly what I thought about the benchmark test. The line:
'Best time for xy bit trial factors' should be read as 'Best time for TF from 2^xy to 2^xy+1' or 'Best time for xy+1 bit trial factors'. Then the CPU-credit for TF with every size of the factors should give (nearly) the same amount of CPU-credit per time.

As far as I understood Jacob correct, every hour of work of any kind should give (nearly) the same CPU-credit. Then I have to conclude that my CPU is simply not good for doing P-1. I will make some tests with other memory settings and so, maybe I can better it.

Thanks again for your answers and Best Regards,

Matthias

I agree entirely with the original poster, and started a thread on that very subject a few days ago that was apparently of little interest:

[url]http://www.mersenneforum.org/showthread.php?t=10920[/url]

I am in this for CPU credit... probably the wrong reason many would say, but since I don't expect to discover too many record primes in my lifetime, there really isn't much else aside from some credit to keep me paying higher electricity bills for years. (I suppose I could keep my electricity bills for posterity, and obtain some satisfaction from gazing at the digits therein...)

The discrepancies in the credit calculations are *very* significant and, like the OP stated above, LL and P-1 testing appear to be the most highly 'penalized'. As such, despite having powerful machines at my disposal, I am not at all incentivized and have been sticking to trial factoring only, which provides approximately 25% more credit over any given time span.

I believe this is a very serious issue, bearing in mind the sheer number of machines running, the number of people participating, and the aim(s) of the project. Credits should not be guesswork: They should be based on sound calculation, taking into account the different algorithms used, CPU instruction cycle timings and so on.

What I have seen on this forum appears to indicate that these numbers are simplistically approximated (to what? 12-digit accuracy?!) with further guesstimated adjustments made as processor technology advances.

Bottom line: If GIMPS would like to encourage non-TF work, or in some cases any work at all (I don't particularly like TF work, and it is beneath my machines' capabilities), then a CPU should get as near as dammit the same amount of credit whether it is doing TF, P-1, LL or ECM. I can understand small discrepancies of a few percent, but 25%+ (even when taking into account RAM bottlenecks) is simply not fathomable.

I won't be doing further non-TF testing until this problem has been addressed -- and the fun of TF is not only wearing a bit thin for me but starting to look pointless with the high exponents already 'TF'd!

[quote=MatWur-S530113;149961]
TF from 2^60 to 2^61: screen output every 45 sec
TF from 2^61 to 2^62: screen output every [B]44[/B] sec
TF from 2^62 to 2^63: screen output every [B]74[/B] sec
TF from 2^63 to 2^64: screen output every 74 sec
TF from 2^64 to 2^65: screen output every 73 sec
[/quote]

The bolded entries look wrong.

I know very little about TF, and have never even done any myself. Mat's benchmark shows plateaus from 60 to 62 and 62 to 65. Is that expected?

[QUOTE=uigrad;150113]Mat's benchmark shows plateaus from 60 to 62 and 62 to 65. Is that expected?[/QUOTE]Yes. There are actually three ranges, below 62 bits, 62 to 64 bits, and above 64 bits, but they sometimes have similar timing results.

[QUOTE=g0vegan;150088]... a CPU should get as near as dammit the same amount of credit whether it is doing TF, P-1, LL or ECM.[/QUOTE]Unfortunately that could only be done by making credit depend on the type of CPU and maybe other characteristics of the machine, as well as the work type & exponent size. I can see the credit system we now have involved trade-offs, so inevitably there were some subjective decisions.

[quote=markr;150149]Unfortunately that could only be done by making credit depend on the type of CPU and maybe other characteristics of the machine, as well as the work type & exponent size. I can see the credit system we now have involved trade-offs, so inevitably there were some subjective decisions.[/quote]

To an extent, yes. But a 64-bit floating point add operation is a 64-bit-floating-point add operation, no matter how it is carried out. You could do it on an abacus over the course of 2 days, or the latest supercomputer in 1 femtosecond: Either way, you have carried out one floating point operation and how long it takes you depends on the technology you are using.

Rather than thinking of credits as they relate to a particular processor (such as a P90, Quad Core 9xxx @ 2.x GHz or whatever), we should think of them in terms of "the number of mathematical operations required in order to perform a type of calculation."

If a calculation involves 1 billion add, subtract, multiply and divide operations, let's not worry about what type of processor carried them out, but simply that they *were* carried out.

Rather than measuring in units of GHz-days, I believe it should be GFlop-days: Credit the user with (for example) the number of 32-, 64- or 128-bit floating point equivalent operations carried out. This would be independent of processor structure, and would simply mean that those with slower processors, less-well-endowed processors, or RAM bottlenecks take longer to perform a "flop".

I am assuming that the current credits are based on benchmark timings -- which are in turn subject to all sorts of different machine-dependent factors. Why should the "GHz-day" of credit that I get be based on some timing performed on a CPU that may have a completely different architecture and motherboard configuration?

By analyzing the algorithms involved, it is possible to very accurately calculate the number of 'flops' required for any given type of computation. For example, it would be quite simple to come up with formula(s) for how many operations are involved in factoring M(p) to 2^n or performing an LL test on M(p) simply by analyzing the algorithm.

I do not believe it is a coincidence that ALL of my computers, each with a completely different architecture -- or the computer(s) belonging to the original poster -- just happen to earn significantly more credit for TF work than other work.