P-1 factoring anyone?

[markr]

Originally Posted by petrw1

I totaled all the LL and P-1 Attempts from the Last 365 Days reports.

LL 81,959
P1 164,981 (twice LL!!!)

NOT SO FAST ...

The P1 total needs work.
I took an educated guess to eliminate all the P1-Small. Any that had a small Points Per of less than 1 with "many" attempts.

New total
P1 92,315 (still a little above LL)

HOWEVER, I suggest we also need to exclude P1 tests that were done on exponents below (or well above - i.e. 100M Digits Project) the current LL line. These would NOT be helping the quest to keep ahead of the immediate LL needs. I do NOT know how to even guess at this number.

I also have no way to determine how many of the P1 were done independantly and how many P1 were done in conjunction with the corresponding LL test but I believe that in either case they are contributing to the current LL needs and should stay included for the sake of this analysis.

Since most of the independent P-1 tests are ahead of the LL wave, I'd bet nearly all the 81,959 LL tests had the P-1 test done by the LL tester. If we subtract that from the estimated 92,315 P-1 tests, we would estimate 10356 "independant" P-1 tests - far less than the number of LL tests. Your analysis also says the the P-1-Small effort has worked through a lot of exponents - a better conclusion!

A few days ago I found some factoring limits data saved from 15 March 2009. Eventually I got around to comparing counts of P-1'd exponents from those files with current data. The time period is a bit shorter than the tidier one petrw1 used, so bear that in mind. This is for ranges 50M to 52M to just reflect results from P-1 assignments (but only those that didn't find a factor).

Number of exponents with P-1 done

Range   2009-03-15  2009-12-30    Change
50M         1946        7445       5499
51M            4        8653       8649
52M            0          96         96
Total       1950       16194      14244

14244 is a bit more than 10356 - probably when you eliminated P-1-Small some P-1-Large were excluded too. Or I did something wrong.

Next, some numbers from summarising some PrimeNet summaries. The ranges (choice of two) are to cover mostly LL activity. The dates are 1/1/2009 for petrw1's start date, 15/3/2009 for my start date, and 31/12/2009 for the present. The differences part looks a bit confusing, sorry.

Totals from 4000000 to 50000000

       01/01/2009  15/03/2009  31/12/2009   Change 01/01  Change 15/03
                                              to 31/12      to 31/12
Count     567480      567480      567480
F         348886      349653      352526          3640          2873
LL-D         220         367        1082           862           715
LL         48359       60073      109629         61270         49556
LLERR       1509        1627        1540            31           -87
NO-LL     168505      155759      102701        -65804        -53058


Totals from 2500000 to 50000000

       01/01/2009  15/03/2009  31/12/2009   Change 01/01  Change 15/03
                                              to 31/12      to 31/12
Count    1435207     1435207     1435207		
F         892982      893844      896942          3960          3098
LL-D        6713        7654       10056          3343          2402
LL        345334      358864      419610         74276         60746
LLERR       5601        4856        2207         -3394         -2649
NO-LL     184572      169984      106386        -78186        -63598

These seem in broad agreement with petrw1's estimate of 81959.

Regardless of how one looks at this data, the conclusion is the P-1 effort is well short of the number of LL tests, so the LL wave is catching up. Not really such a surprise, I guess, except for the size of the difference. It doesn't change the point - each P-1 test done means the final stage of TF goes to the over-resourced TF queue instead of slowing the LL tester.

[garo]

Good analysis! I think the discrepancy of 4K can be explained by the fact that a small but significant proportion of LL testers do not do P-1 before testing even when the exponent has not been P-1 tested previously. Take 45000161 as an example.

Of course all this does not take into account the fact that many P-1 tests which are done by LL testers are stage-1 only. My thoroughly unscientific sample is showing that only about 55-60% of exponents in the 45M range that have had an LL test done got a full P-1 (i.e. stage 1 & 2). About a third get a stage-1 only P-1 and about 10% get no P-1 at all.

[petrw1]

Originally Posted by markr

Since most of the independent P-1 tests are ahead of the LL wave, I'd bet nearly all the 81,959 LL tests had the P-1 test done by the LL tester.

Forgot about that and once we run out of exponents that only need P-1 and LL then P-1 will fall behind.

Originally Posted by markr

14244 is a bit more than 10356 - probably when you eliminated P-1-Small some P-1-Large were excluded too. Or I did something wrong.

70,000 of the 73,000 P-1-Small were from 5 testers with BIG counts: Carsten(55,000), Markr(1,300), Rob(3,000), Harlee(9,000), AXN(1700) so I don't think I should be out by much.

Originally Posted by markr

Regardless of how one looks at this data, the conclusion is the P-1 effort is well short of the number of LL tests, so the LL wave is catching up. Not really such a surprise, I guess, except for the size of the difference. It doesn't change the point - each P-1 test done means the final stage of TF goes to the over-resourced TF queue instead of slowing the LL tester.

Considering a P-1 test takes about 1/15th the time of the LL time we only need 1 P-1'er for each 15 LL'ers.

Originally Posted by Garo

Of course all this does not take into account the fact that many P-1 tests which are done by LL testers are stage-1 only. My thoroughly unscientific sample is showing that only about 55-60% of exponents in the 45M range that have had an LL test done got a full P-1 (i.e. stage 1 & 2). About a third get a stage-1 only P-1 and about 10% get no P-1 at all.

Not sure if these are the same stats but from the Factoring Limits report
21,501 Exponents in 45M range
14,169 P-1 done - 66%
5,878 Stage 1 only - 41% of P-1

Conversely in the 50M range where P-1 has been independently assigned:
21,424 Exponents in 50M range
7,448 P-1 done - 35%
7 (yes, seven) Stage 1 only - but very high stage 1 (800,000-980,000)
Virtually all non P-1'd are already at the required 69 bits of TF


I guess the dark horse is that if/when necessary George can change the "whatever makes sense" option to get more P-1 done.

[garo]

Originally Posted by petrw1

Not sure if these are the same stats but from the Factoring Limits report
21,501 Exponents in 45M range
14,169 P-1 done - 66%
5,878 Stage 1 only - 41% of P-1

I was only counting exponents that are not currently assigned. Still, we can conclude that about 30-40% of P-1s done by the LL tester are stage 1 only.

[markr]

Originally Posted by garo

Good analysis! I think the discrepancy of 4K can be explained by the fact that a small but significant proportion of LL testers do not do P-1 before testing even when the exponent has not been P-1 tested previously. Take 45000161 as an example.

Of course all this does not take into account the fact that many P-1 tests which are done by LL testers are stage-1 only. My thoroughly unscientific sample is showing that only about 55-60% of exponents in the 45M range that have had an LL test done got a full P-1 (i.e. stage 1 & 2). About a third get a stage-1 only P-1 and about 10% get no P-1 at all.

Right! Adjust for the difference in time periods and reduce the number of P-1 done as part of LL by 10%, and there's basically no difference with petrw1's estimates eliminating P-1-Small.

I expected all sorts of reasons for differences, eg different time periods, not counting successful P-1. Still, one can compare some aspects of apples & oranges - but not granny smiths (yum) & delicious (not).

Trying to get a "number of LL tests done" out of the PrimeNet summaries was a problem I left for another time (change in LL-D + change in LL looks close) since there was no need at the moment, the gap between P-1 & LL done was so great.

[cheesehead]

Originally Posted by petrw1

I guess the dark horse is that if/when necessary George can change the "whatever makes sense" option to get more P-1 done.

That could be an interesting PrimeNet assignments experiment for a couple of days.

It could persuade some folks to change "whatever makes sense" to "that which I want to do".

[Brian-E]

Originally Posted by cheesehead

That could be an interesting PrimeNet assignments experiment for a couple of days.

It could persuade some folks to change "whatever makes sense" to "that which I want to do".

Maybe, but in my case at least it has recently worked the other way: after over 2 years of taking part with "whatever makes sense" I have become frustrated after being given TF assignments continually for many months when I read that these are over-subscribed and not what is needed to help the project. So two months ago I changed to a preference for P-1 (large), and recently changed again to a preference for Double Checking (since that is also apparently in need).

If and when "whatever makes sense" is altered to truly reflect the needs of the project, I will revert to that option.

[cheesehead]

Originally Posted by Brian-E

Maybe, but in my case at least it has recently worked the other way

I meant to suggest that folks who currently specify "whatever makes sense" would change their preferences to specific types such as LL, DC, P-1 ... (the latter is what I meant by "that which I want to do") ...

So two months ago I changed to a preference for P-1 (large), and recently changed again to a preference for Double Checking

... which is just what you seem to have done.

after over 2 years of taking part with "whatever makes sense" I have become frustrated after being given TF assignments continually for many months when I read that these are over-subscribed and not what is needed to help the project.

You know that PrimeNet's "whatever makes sense" doesn't mean simply "whichever category is most under-resourced from a forum participant's POV", don't you? IIRC it takes other factors into account, such as your CPU type and whatever-that-rating-of-how-PrimeNet-thinks-you've-done-so-far-is-called (not reliability, but something like that) in addition to George's fudge factors.

When I've specified "whatever makes sense", I've never been given TF assignments, but maybe if I were still using my mid-1990s P75 to request assignments ...

[markr]

Originally Posted by cheesehead

When I've specified "whatever makes sense", I've never been given TF assignments, but maybe if I were still using my mid-1990s P75 to request assignments ...

Your P75 would quite possibly only get LMH work - its "Pentium 4 equivalent speed" could well be less than the current 50MHz threshold. To avoid TF by default it would need to be much faster: the minimum "Pentium 4 equivalent speed" to get double-check tests is currently 800 MHz. And if you specify & run it less than 24 hours per day the actual speed needed is higher still.

[cheesehead]

Originally Posted by markr

Your P75 would quite possibly only get LMH work

That's just TF at high exponents, which is what I had in mind and would have expected.

Despite the "Account-Level Work Type Preference" subcategory of TF (http://www.mersenne.org/worktype/) for LMH, there's no "PrimeNet Assignment Work Type" (http://www.mersenne.org/worktypes/) or "PrimeNet Stats Type" http://www.mersenne.org/statstypes/) category that separates LMH from other TF, is there?

:-)

[markr]

Originally Posted by cheesehead

Despite the "Account-Level Work Type Preference" subcategory of TF (http://www.mersenne.org/worktype/) for LMH, there's no "PrimeNet Assignment Work Type" (http://www.mersenne.org/worktypes/) or "PrimeNet Stats Type" http://www.mersenne.org/statstypes/) category that separates LMH from other TF, is there?

:-)

One could argue TF-LMH is a category of work preference, according to the "Account-Level Work Type Preference" page (http://www.mersenne.org/worktype/) and the PrimeNet web API http://v5.mersenne.org/v5design/v5webAPI_0.97.html#7.0. Another example is the varieties of LL test available (world record, 10M digits, an so on). Some of the work preference categories are combined to form stats categories. (A complication for some discussion topics is that some of the project's web pages may not be entirely current or accurate. Just a general comment.)

Sometimes "TF" means "all TF", sometimes it means "non-LMH TF". I jumped to a conclusion, but not very far. :)

Originally Posted by Lewis Carroll in "Through the Looking Glass"

"When I use a word," Humpty Dumpty said in rather a scornful tone, "it means just what I choose it to mean -- neither more nor less."
"The question is," said Alice, "whether you can make words mean so many different things."
"The question is," said Humpty Dumpty, "which is to be master - - that's all."

(Among Carroll's many entertaining quotes, that one is a favourite among people who have been careless with words. It's often mis-attributed to the Red Queen, in "Alice in Wonderland".)