Relative speeds of hardware for different types of work

[S00113]

What are the relative speeds of different types of CPU/cache/memory combinations on different types of work?

o What is the maximum amount of memory that will be used on stage 2 of P-1 on different CPUs and ranges?
o At what cahce sizes do different FFT sizes "break down" and hit the memory access bottleneck?
o How many cycles do each kind of CPU use on each iteration of
- factoring (to how many bits?)
- LL testing (different FFTs)
- P-1 testing (both stages)

My .ini files are genreated automatically based on information about CPU and memory, and I would like each machine to be as productive as possible. Time is not a major problem, as I back up all changed files every second hour and complete the work on another machine if the first one breaks down. Memory use is not a problem either, since I shut down mprime when a user logs on. (I don't know how to do this on Windows machines, so I use them for factoring only.)

Are there plans to implement smarter work distribution in the next version of the server? I.e. if P4s are very effective for factoring up to 64 bits, but less effective to 65 bits, then P4s used for factoring should return their exponent to the server at 64 bits and get a new one, while the first is sent to a PIII or earlier for factoring to 65 bits or above.

[dsouza123]

For trial factoring:

P4s are very slow at 64 bits and below but very quick at 65 and higher (SSE2 used instead of FPU).

For 64 bits and below, at the same clock speed 1 Ghz,
Athlon is considerably faster, PIII somewhat faster and the P4 extremely slow.

A P4 at a sufficiently high clock speed will still be OK because alot of the time
is spent at 65 bits and above using the SSE2 instructions.

[Mr. P-1]

LL testing, Doublechecking, and P-1 are all basically the same kind of work, i.e., they use the FP FFT transform to do huge multiplications. A processor which is good at one of them will be equally good at the others. Trial Factoring uses only integer arithmetic, except on P4s and recent Celerons, which use FP arithmetic (in a completely different way from FFT) over 64 bits.

P4s and recent Celerons are best at FFT work. They're OK for trial factoring above 64 bits, lousy below 64 bits.

Athlons and Durons and P3s are OK at everything.

Older processors are best at trial factoring.

A reasonable amount of cache is important for FFT work. Not sure how important it is for TF.

Large amounts of memory are used only during stage 2 of P-1. How much is actually used will depend upon the exponant, and the amount you make available. Quite a complex algorithm is used to decide how much to use. Basically if the optimal amount is larger than the available amount (Very likely, because the optimal amount is it the GBs for exponants currently being tested) then it repeatedly decrements the amount until it fits. The amount by which it decrements is very large - tens or even hundreds of MBs. Also, the client is programmed not to go right up to the limit you set.

The upshot of all this is that it will use quite a bit less than you allow.

[cheesehead]

Originally Posted by S00113

o At what cahce sizes do different FFT sizes "break down" and hit the memory access bottleneck?

It's not really a question of cache sizes at which FFT sizes hit memory access bottlenecks.

Prime95 takes the L1 cache size into account and structures its memory accesses to maximize throughput accordingly. That is, if L1 cache size is 8KB then Prime95 will completely process an 8KB slice of the FFT before it processes the next 8KB slice of the FFT. Furthermore, it "touches" blocks of memory ahead of when it will perform arithmetic on their contents, so that those blocks will be prefetched into cache while Prime95 is processing something else. So in this regard Prime95 already optimizes memory accesses, and FFT size doesn't matter much to that.

(And everything above about Prime95 also applies to mprime.)

I would like each machine to be as productive as possible.

Okay, but keep in mind that in some cases the differences in productivity (between different combinations of CPU and GIMPS work assignment types) are relatively minor, and forum participants sometimes make more of these differences than they deserve.

[cheesehead]

Originally Posted by S00113

I.e. if P4s are very effective for factoring up to 64 bits, but less effective to 65 bits, then P4s used for factoring should return their exponent to the server at 64 bits and get a new one, while the first is sent to a PIII or earlier for factoring to 65 bits or above.

As pointed out earlier, P4 trial-factoring efficiency is the other way around. But in either case, one needs to take into account the inefficiency created by splitting a factoring assignment into two parts -- twice the assignment management and communication overhead, more opportunity for mistakes, and so on. Finally, note that for numbers TFed to more than 2^64, the time spent on TF to 2^64 will be only a fraction of the time spent on TF above 2^64, so whatever inefficiency there is for TF below 2^64, it's only a small part of the total effort.

[S00113]

Originally Posted by cheesehead

As pointed out earlier, P4 trial-factoring efficiency is the other way around. But in either case, one needs to take into account the inefficiency created by splitting a factoring assignment into two parts -- twice the assignment management and communication overhead, more opportunity for mistakes, and so on. Finally, note that for numbers TFed to more than 2^64, the time spent on TF to 2^64 will be only a fraction of the time spent on TF above 2^64, so whatever inefficiency there is for TF below 2^64, it's only a small part of the total effort.

I think it still would be an improvement. Windows machines are reinstalled very often. When one of my factoring Windows machines report progress, it would be nice if the amount of factoring done on that expoment was recorded. As of today, even if the machine had factored an exponent from 2^60 to 2^67, the report still says it is at 2^60 until factoring to 2^68 is complete. If this is an old Pentium, factoring to 2^68 will most likely never be completed until the machine is retired because it is so inefficient at this task. A Pentium 4 would complete it in less than a day.

Progressive reporting of factoring is practically done already, and less work would have to be repeated. When progressive reporting is implemented, it would be a simple task to put the exponent back in line after factoring to a certain limit.

Similar could be done with stages 1 and 2 of P-1 factoring. Many expoments will never get a stage 2 run. I have a few fast machines with GBs of memory dedicated to P-1 factoring, and it would be best if I could get P-1 stage 2 work directly from the server. For now I allocate lots of work with SequentialWorkToDo=0 and move the Test= lines to machines with less memory when P-1 is done.

The simple task is to divide P-1 factoring and trial factoring into two different work types, stages 1 and 2 and <64 bit and >64 bit, and let mprime choose the work somewhat intelligently.

[Complex33]

If you have a concern about partial factoring, you know you can manipulate primenet to perform factoring up to whatever bit depth you desire and then dump the results back to the server. After a while a script comes behind the returned work that becomes "stuck" and puts the exponents back on the market at the new bit depth.

[cheesehead]

Originally Posted by S00113

As of today, even if the machine had factored an exponent from 2^60 to 2^67, the report still says it is at 2^60 until factoring to 2^68 is complete. If this is an old Pentium, factoring to 2^68 will most likely never be completed until the machine is retired because it is so inefficient at this task. A Pentium 4 would complete it in less than a day.

Progressive reporting of factoring is practically done already, and less work would have to be repeated. When progressive reporting is implemented, it would be a simple task to put the exponent back in line after factoring to a certain limit.

Oh, so what you want is a systematic way of reporting the partial completion of a TF assignment in case the machine can't complete the assignment, along with automatic reassignment of the remaining portion. Well, that's certainly worthwhile.

What I was referring to was the deliberate splitting-up of assignments even though the machine was fully capable of completing the entire un-split factoring range. Your proposal refers, in effect, to an unplanned splitting of an assignment (though once it's implemented, it could also be used with deliberate intent).

Similar could be done with stages 1 and 2 of P-1 factoring. Many expoments will never get a stage 2 run. I have a few fast machines with GBs of memory dedicated to P-1 factoring, and it would be best if I could get P-1 stage 2 work directly from the server.

Yeah, but stage 2 has to start with the save file (4MB in size, for a 32M exponent, e.g.) from the preceding stage 1. So you've got a noticeable (for dialup users, anyway) extra communication overhead, which the stage 1 assignee has to pay, too, when reporting his/her results.