Sierp base 63 - team drive #5
 

I have been running some tests to estimate the effort involved for S63. I would like to get it to n=10000. Based upon my tests I estimate that close to 70% of the k (of 237,036) have primes for n < 10000. Based upon my tests I also estimate that testing 10000 k to n=10000 will take about 2 months on a single core of a C2D at 2.4 GHz.

I used srsieve to sieve to a depth of 1e10. I then took the ABC file that srsieve output (using the -w switch) and modified the first line to read as follows:

ABC $a*63^$b$c // {number_primes,$a,1}

I then ran the file (sr_63.pfgw) through PFGW without any additional switches. In my test I only did 100 k, so my test results are an extrapolation. You can certainly use sr2sieve, but you would have to break up the one file into multiple files. It might save some time sieving, but takes more time to manage.

I have created 23 files, each with 10,000 k (I've already taken the last file with 7036 k). To join in this effort, please select a file from one of the following links and state your reservation into this thread. When you have completed processing the file, please zip up the primes you have found and attach them to a post in this thread. Based upon the success of this thread I might do a second drive in the future.

Group 1 (complete by Mathew Steine; 2444 k's remaining)
Group 2 (complete by Lennart; 2381 k's remaining)
Group 3 (complete by Lennart; 2419 k's remaining)
Group 4 (complete by Lennart; 2343 k's remaining)
Group 5 (complete by Lennart; 2365 k's remaining)
Group 6 (complete by Lennart; 2360 k's remaining)
Group 7 (complete by rogue; 2456 k's remaining)
Group 8 (complete by Xentar; 2322 k's remaining)
Group 9 (complete by Lennart; 2384 k's remaining)
Group 10 (complete by rogue; 2393 k's remaining)
Group 11 (complete by appeldorff; 2368 k's remaining)
Group 12 (complete by Mini-Geek; 2437 k's remaining)
Group 13 (complete by rogue; 2394 k's remaining)
Group 14 (complete by rogue; 2462 k's remaining)
Group 15 (complete by kar_bon; 2369 k's remaining)
Group 16 (complete by Lennart; 2359 k's remaining)
Group 17 (complete by rogue; 2327 k's remaining)
Group 18 (complete by rogue; 2389 k's remaining)
Group 19 (complete by rogue; 2348 k's remaining)
Group 20 (complete by rogue; 2379 k's remaining)
Group 21 (complete by rogue; 2427 k's remaining)
Group 22 (complete by rogue; 2344 k's remaining)
Group 23 (complete by rogue; 2380 k's remaining)
Group 24 (complete by rogue; 1698 k's remaining)

Any comments or suggestions are welcome.

[B]The drive is now complete! 180488 primes were found. 56548 k's remain at n=10K. 76.14% of all k's were eliminated for n=1K-10K![/B]

I was playing around with group 1, (not reserving it) and I have a very useful (though maybe obvious) tip for anybody planning on doing this: when starting out the sieve, make srsieve quiet by setting -m to the same as -P. Otherwise, when srsieve hits -m's value (default 100K), it slows down tremendously trying to print every one of the millions of factors found.

Also, sr2sieve is impractical for very large k's (e.g. everything except group 1) and large groups of k's (e.g. any of these groups unless you split them up). Even in group 1, you'd definitely want to only spend the half hour to generate the Legendre tables once (run "sr2sieve -c -i sr_63.abcd" once, it'll save it to sr2cache.bin where all other sr2sieve runnings will automatically look). Even with the first 1000 k's, it took 120 MB of RAM to run, so it might not be practical, depending on how it behaves with all 10000 k's and your RAM limits. I'm not sure how good sr2sieve -x (no Legendre lookup) would be.

Cool idea! However, to avoid confusion with the [URL="http://www.mersenneforum.org/showthread.php?t=9822"]"real" Drive #1[/URL] (S16), I suggest that we rename this to either:
"Sierpinski base 63 - team drive #4"
"Sierpinski base 63 - mini-drive 1"
per our established naming conventions for team efforts. Since this effort, at my rough approximation, will take about 4 CPU-years, I'm not sure it exactly fits in the category of a mini-drive, and am thinking that designating it team drive #4 would be more appropriate. Gary, what do you think?

I take [URL="http://home.roadrunner.com/%7Emrodenkirch/group02.zip"]Group 2[/URL]

Lennart

[quote=mdettweiler;211736]Cool idea! However, to avoid confusion with the [URL="http://www.mersenneforum.org/showthread.php?t=9822"]"real" Drive #1[/URL] (S16), I suggest that we rename this to either:
"Sierpinski base 63 - team drive #4"
"Sierpinski base 63 - mini-drive 1"
per our established naming conventions for team efforts. Since this effort, at my rough approximation, will take about 4 CPU-years, I'm not sure it exactly fits in the category of a mini-drive, and am thinking that designating it team drive #4 would be more appropriate. Gary, what do you think?[/quote]

Yeah, it probably needs to be something besides team drive 1.

The mini-drive designation applies to only the base itself and would be appropriate if we were doing only a portion of the k's with this drive like we did for base 3. But since we're doing all of them, then this wouldn't be considered a mini-drive.

Once complete to n=10K, then this drive could be simply extended to n=15K or 25K like we've done for base 16. Otherwise if we made it another team drive for n>10K, we'd have to have another thread and would ultimately innundate ourselves with threads and drives. We've kind of done that already, especially for base 3, which is why I unstickied all of those long ago. :-)

Therefore I think team drive #4 is the way to go.

Thoughts from anyone else on it?


Gary

[quote=Mini-Geek;211734]I was playing around with group 1, (not reserving it) and I have a very useful (though maybe obvious) tip for anybody planning on doing this: when starting out the sieve, make srsieve quiet by setting -m to the same as -P. Otherwise, when srsieve hits -m's value (default 100K), it slows down tremendously trying to print every one of the millions of factors found.

Also, sr2sieve is impractical for very large k's (e.g. everything except group 1) and large groups of k's (e.g. any of these groups unless you split them up). Even in group 1, you'd definitely want to only spend the half hour to generate the Legendre tables once (run "sr2sieve -c -i sr_63.abcd" once, it'll save it to sr2cache.bin where all other sr2sieve runnings will automatically look). Even with the first 1000 k's, it took 120 MB of RAM to run, so it might not be practical, depending on how it behaves with all 10000 k's and your RAM limits. I'm not sure how good sr2sieve -x (no Legendre lookup) would be.[/quote]

Agreed on every front here. If doing 10,000 k's at once for k's of this size, you have to use srsieve or sr2sieve without the symbols, i.e. with the -x switch. I believe that sr2sieve has an inherent limitation itself on memory allocation, although I could be wrong. I seem to recall butting up against a memalloc error even when my machine had far more memory than when the error occurred. Regardless, even the first 10,000 k's would likely eat 5-10 GB of memory or more and take days or even weeks to create the symbols. For such a low n-range, it isn't worth it.

If anyone would care to post their timings on srsieve vs. sr2sieve with the -x option, that would be helpful.


Gary

[QUOTE=gd_barnes;211740]Yeah, it probably needs to be something besides team drive 1.

The mini-drive designation applies to only the base itself and would be appropriate if we were doing only a portion of the k's with this drive like we did for base 3. But since we're doing all of them, then this wouldn't be considered a mini-drive.

Once complete to n=10K, then this drive could be simply extended to n=15K or 25K like we've done for base 16. Otherwise if we made it another team drive for n>10K, we'd have to have another thread and would ultimately innundate ourselves with threads and drives. We've kind of done that already, especially for base 3, which is why I unstickied all of those long ago. :-)

Therefore I think team drive #4 is the way to go.

Thoughts from anyone else on it?[/QUOTE]

Gary, since you have admin access, feel free to change the name of the thread to the consensus.

I take
[URL="http://home.roadrunner.com/%7Emrodenkirch/group03.zip"]Group 3[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group04.zip"]Group 4[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group05.zip"]Group 5[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group06.zip"]Group 6[/URL]


Lennart

I have done some testing and can state that for the range I'm doing that srsieve is faster than sr2sieve.

Using srsieve with -P and -m, I am getting about 40000 p/s (1500 k's in the file).

Using sr2sieve with -P, -q, and -x, I am getting about 25000 p/s on the same file.

1e10 is not be the optimal sieve depth. It is a ballpark number. In reality it could be more or less dependent upon the number of k's in the input file. In my original test I had 100 k's (the lowest 100), but I also used sr2sieve. 1e10 was slightly deeper than necessary, but not by much.

OK, I've changed it to team drive #5. I've also showed Lennart's reservations in the 1st post.

Does anyone want group 1?

Since larger k take longer to test, so my estimate of two months is most likely incorrect. I have split group 24 (not shown in this thread) across 4 CPUs (2 of which are 'borrowed" for a few days) and will be able to provide a better estimate in a few weeks.

I have a hacked (cough) version of srfile which can take a file argument with -d. This allows one to use the following command:

srfile -w sr_63.pfgw -d pfgw.log

to remove all k with found primes with one call to srfile. Since sr_63.pfgw can be very large (depending upon the number of k you have in it), this saves a lot of I/O (and time) if you alternate between PRP testing and sieving.

If interested, PM me and I can send you the code changes or a Windows exe. I'll send my changes to Geoff. Hopefully he will incorporate them in his next release.

Lennart can't have all the fun
 

I would like to reserve group 1

In a manner similar to the way I double checked Riesel base 3 for n<=25K when KEP reserved a large range for n=25K-100K, I have double checked all k's remaining on S63 for k<14M for n<=1K so far. Just like R3 for the k=500M-820M range, I found 2 problems. They are:

Both k=9632032 and 12224450 still remain.

I attempted to quickly resolve the situation by searching them to n=5K without sieving. One went down:

9632032*63^1612+1 is prime

So fortunately k=9632032 (the group that would have contained it is reserved by Lennart) is now not a problem but k=12224450 will need to be added to the appropriate group here or done separately. Mark, it's up to you how it gets addressed. Fortunately it has not been reserved yet.

Kenneth, can you check your primes list for both of these two k's and see what you have for them? I'm wondering why you did not have them remaining. Neither are MOB so that confusion (since your search was done before the current starting bases script was done) is not a possibility. Based on finding these 2 problems, I suspect there will be more. Therefore I will slowly continue my double check and attempt to stay ahead of the reservations.

All-in-all, this really isn't bad at all. On both R3 and S63, I had expected to find many more problems than I did because the ranges that I double checked were done before we had the automated starting bases script. It was not easy at all on these huge-conjectured new bases to get everything right before we finally got the starting bases script set up mostly correctly.

On another note, I went ahead and gave this thread the visibility it deserves and stickied it. Can anyone say 2 pages of stickies? :-)

Anyone wondering why admining this stuff takes a little time? :smile:


Gary

[QUOTE=gd_barnes;213195]Kenneth, can you check your primes list for both of these two k's and see what you have for them? I'm wondering why you did not have them remaining. Neither are MOB so that confusion (since your search was done before the current starting bases script was done) is not a possibility. Based on finding these 2 problems, I suspect there will be more. Therefore I will slowly continue my double check and attempt to stay ahead of the reservations.[/QUOTE]

That is unfortunantly not possible, since I have no primes for S63 remaining in my system anymore. But I remember that when I started it up, I used the script Henryzz had written to start up base 3. I used the excel spreadsheet to remove the k's remaining with trivial factors, and since that method has previously caused problems, these 2 k's might have been missed because of a miscalculation in excel. But if I had the primes still, I of course wouldn't have minded taking a look at these 2 k's you have found. You think you can find 4 more before the end of your doublecheck :wink:? :smile:

Kenneth

It just crossed my mind, that I actually had you download the n=1-1000 primelist for S63, from my adrive account. Maybe you can retrieve the list and see if there is in fact primes for the 2 k's mentioned. Or did you never reach to download it (though I recall it as you did)?

Kenneth

[quote=KEP;213237]It just crossed my mind, that I actually had you download the n=1-1000 primelist for S63, from my adrive account. Maybe you can retrieve the list and see if there is in fact primes for the 2 k's mentioned. Or did you never reach to download it (though I recall it as you did)?

Kenneth[/quote]

No, I was having too much problem downloading it. It was just taking too long to download.

That was another reason that I'm doing the doublecheck...so that I could have a comprehensive list of all of the S63 primes. It's what I eventually do on all of the large-conjectured bases since it would be unreasonable and ineffective to have people forward me millions of very small primes.

There's no telling how many more problems that I will find. It could be none or it could be 10 or more or it could be 4. I was pretty impressed by the time I got to k=9M with no problems. I was beginning to think there was going to be none.

Mark, be sure and add k=12224450 to the appropriate group here if that is how you choose to. I suppose someone could just search it from n=5K-10K. We would then just have to remember that it is remaining even though it's not in the k's that we're searching with this drive.

[QUOTE=gd_barnes;213275]No, I was having too much problem downloading it. It was just taking too long to download.

That was another reason that I'm doing the doublecheck...so that I could have a comprehensive list of all of the S63 primes. It's what I eventually do on all of the large-conjectured bases since it would be unreasonable and ineffective to have people forward me millions of very small primes.

There's no telling how many more problems that I will find. It could be none or it could be 10 or more or it could be 4. I was pretty impressed by the time I got to k=9M with no problems. I was beginning to think there was going to be none.

Mark, be sure and add k=12224450 to the appropriate group here if that is how you choose to. I suppose someone could just search it from n=5K-10K. We would then just have to remember that it is remaining even though it's not in the k's that we're searching with this drive.[/QUOTE]

I will wait until you have completed your check and I will decide what to do at that time.

Primality testing 12224450*63^6893+1 [N-1, Brillhart-Lehmer-Selfridge]
Running N-1 test using base 3
Calling Brillhart-Lehmer-Selfridge with factored part 53.00%
12224450*63^6893+1 is prime! (34.7063s+0.0082s)

Sure right after rogue's post

Good one Mathew. It figures that the prime would be just a little bit larger than the n-limit that I chose to search to. :smile:

When I update the pages for S63, I'll show just the 2 primes on the k's remaining that were missed since I'll keep the # of k's remaining the same.

[quote=rogue;213292]I will wait until you have completed your check and I will decide what to do at that time.[/quote]

It could be 3-4 weeks before I'm done. I only have it running k=1M at a time on and off on my slow dual-core 1.6 Ghz laptop that I have with me on business trips. Based on that, I think I may put a single speedy I7 core on it. Even though it would only be one core, since it's running nearly 2X faster and I'll just set it to run non-stop, it should complete sooner; perhaps 2-3 weeks. I'll get that started after I get home...probably early Fri. morning.


Gary

[QUOTE=gd_barnes;213423]It could be 3-4 weeks before I'm done. I only have it running k=1M at a time on and off on my slow dual-core 1.6 Ghz laptop that I have with me on business trips. Based on that, I think I may put a single speedy I7 core on it. Even though it would only be one core, since it's running nearly 2X faster and I'll just set it to run non-stop, it should complete sooner; perhaps 2-3 weeks. I'll get that started after I get home...probably early Fri. morning.[/QUOTE]

I'm in no hurry.

Group 24 results
 

I had this group spread across four cores. It finished overnight. I have revised my estimate for the time needed for a single group to about three months on a single core of a Core 2 Duo at 2.4 GHz. The results are attached. I ask that everyone add the group name to their files before you submit your results.

Here is a summary:

7036 starting k
5338 primes found
1698 k remaining

This removed about 75.87% of the ks. I estimate that once all k reach n=10000 that there will be about 57,204 k remaining.

[quote=rogue;213503]I had this group spread across four cores. It finished overnight. I have revised my estimate for the time needed for a single group to about three months on a single core of a Core 2 Duo at 2.4 GHz. The results are attached. I ask that everyone add the group name to their files before you submit your results.[/quote]
Maybe I'm misunderstanding something, but what quad can run about 2 C2D-2.4GHz months (7036/10000 * 3 months) overnight? Each core would need to run equivalent to about a 432 GHz C2D core (there's 1440 hours in 60 days, figuring 8 hours is "overnight", that's 1440/8=180 times faster; 180*2.4=432). I know this is probably excluding sieving, which is probably significant, but optimal sieving is (almost?) always much shorter than the test, so it shouldn't affect that too significantly.
On rereading, did you mean that it finished overnight but was started some time ago? I took it to mean that it ran and finished overnight. How long did it take 4 cores to complete your group? How long did the sieving take?

[QUOTE=Mini-Geek;213507]Maybe I'm misunderstanding something, but what quad can run about 2 C2D-2.4GHz months (7036/10000 * 3 months) overnight? Each core would need to run equivalent to about a 432 GHz C2D core (there's 1440 hours in 60 days, figuring 8 hours is "overnight", that's 1440/8=180 times faster; 180*2.4=432). I know this is probably excluding sieving, which is probably significant, but optimal sieving is (almost?) always much shorter than the test, so it shouldn't affect that too significantly.
On rereading, did you mean that it finished overnight but was started some time ago? I took it to mean that it ran and finished overnight. How long did it take 4 cores to complete your group? How long did the sieving take?[/QUOTE]

I started two weeks ago (on the 14th). I split the 7036 k into four files. I sieved each file to about 4e9, which took about 30 hours. Extrapolating that to 2500 per file (from about 1750) implies that sieving should take about 42 hours, although sieving longer (48 hours) probably won't have a significant impact. PRP testing started Friday, about two weeks ago. The cores finished through the week, but at different times because I didn't split exactly at 7036/4. The smallest file took about 12 days and the largest took about 15 days. The smallest file contained a little more than 1400 k, which is about 1/7th of 10,000. This should yield around 84 days, but presuming users might have processes stealing cycles and other potential maintenance, I would extend that to about 90 days. Clearly it could be done in less time, but I would prefer to over-estimate the effort rather than under-estimate it.

Group02
 

Here are the primes from group02

7670 Primes

Lennart

Edit:
There are 48 k's with 2 primes each and 1 k with 4 primes so there are 7619 k's with primes and 2381 k's remaining for this group.

[quote=Lennart;213541]Here are the primes from group02

7670 Primes

Lennart[/quote]

Very good and close to Mark's prime percentage...76.7% vs. 75.87%. With that verification, ~57K k's remaining at n=10K as shown by Mark looks like a good estimate.

This is not as "prime" of a base as I had hoped for for a 2^q-1 base. The above estimate would be like 57 k's remaining for a conjecture of k=37K at n=10K. That's good but not great for this size of base. Base 31 is definitely better for its base and conjecture size. It would be like 11 k's remaining for a conjecture of k=64K at n=25K; one of the better ones I've seen. Base 3 has no peer in that regard.

I found another k that is actually remaining at n=1000 but wasn't in the original file of k's remaining. It is k=17131266. Fortunately I was able to eliminate it:

17131266*63^3205+1 is prime

So that now makes 3 k's remaining at n=1000 that weren't in the original file but all of them have primes found for n>1000. That means they don't have to be somehow added to this drive later on.

For my n<=1000 doublecheck, I'm now at k=18M and continuing on until I've finished all k's.

[quote=Lennart;213541]Here are the primes from group02

7670 Primes

Lennart[/quote]

Lennart,

To save me some time later on, could you forward or post the k's remaining in this group?

Mark,

Since you're administering the effort, if you have an easy way to take the primes from each person and remove them from the k's remaining at n=1K to get the k's remaining at n=10K, then feel free to. If you do that, sending a large continguous range of k's remaining to me after several contiguous groups have been completed would work the best for me; perhaps after groups 1 thru 6 have been completed.


Thanks,
Gary

[QUOTE=gd_barnes;213950]
Since you're administering the effort, if you have an easy way to take the primes from each person and remove them from the k's remaining at n=1K to get the k's remaining at n=10K, then feel free to. If you do that, sending a large continguous range of k's remaining to me after several contiguous groups have been completed would work the best for me; perhaps after groups 1 thru 6 have been completed.[/QUOTE]

That will be easy to do. This is what I do:

1) Take the file I posted and strip everything from '*' and after.
2) Take the file of primes found and do the same.
3) Sort each file by k.
4) diff the two files and redirect to another file.
5) Verify that new file only has added lines in the first file and no added lines in the second file. If this step fails, I need to investigate.
6) Sort the new file and strip out extraneous information.
7) Count the number of line in each file (obviously 1 k per line)
8) If "lines in original" - "primes found" = "lines from diff", then I'm good to go and the output from step 6 is the remaining k for the range.

Most of these things are easy to do with TextPad, except diff. I use the command line diff from CygWin because TextPad diff doesn't work very well. TextPad does have the ability to remove duplicate lines when sorting. I do that with step 3. AFAIAC, if there are two or more primes for one k, then it will eliminate the duplicate k so that each k appears once in the file.

Sierp 63 G03 done 7582 primes

Edit:
1 k-value had 2 primes so there are 7581 k's with primes and 2419 k's remaining for this group.

The k and primes are:
k=4892176 for n=5812 and 5948

Sierp 63 G05 done 7636 primes


Lennart

Edit:
1 k-value had 2 primes so there are 7635 k's with primes and 2365 k's remaining for this group.

The k and primes are:
k=8169440 for n=5234 and 5236

Sierp 63 G04 done 7702 primes

Lennart

Edit:
There are 45 k's with 2 primes each so there are 7657 k's with primes and 2343 k's remaining for this group.

Sierp 63 G06 done 7643 primes

Lennart

Edit:
There are 3 k's that have 2 primes each so there are 7640 k's with primes and 2360 k's remaining for this group.

The k's and primes are:
k=9313088 for n=1966 and 1967
k=9474514 for n=2185 and 2186
k=9832868 for n=2003 and 3267

Taking Group 7.

Come on guys. We're almost a third the way done with this drive.

Some good news: I've now finished a complete doublecheck for all 37M+ k's to n=1000. No additional problems were found.

I think KEP did an excellent job running all k's to n=1000. Only 3 discrepencies were noted out of 237K+ k's remaining without any kind of starting script. All 3 k's had primes for n=1K-10K so no adjustments need to be made to the drive. Very nice. :smile:

I'd like to ask you Mark: After you have finished sieving, how are you running your ranges here? I'm assuming that 5M+ k/n pairs are too many for a single PRPnet server. Are you using multiple PRPnet servers running concerruntely or are you reloading the same server and running them in consecutive succession a 1000 or so k's at a time? Or can a single server handle such a huge load or are you doing something completely different? So far, I've only run clients on the 3.2.5/6 servers. I haven't attempted to load one with pairs yet.

I guess the above leads to another question: How many pairs can a PRPnet server handle at one time? Is that related to the memory of the machine or is there a server memory or pairs limit?

This would be a truly extreme stress test for a PRPnet server. I've been running PRPnet servers on port 9000 at NPLB and a personal port for my base 26 effort here but both have large tests...nothing that would stress it much. I sure like the output on the clients now and I've not had any problems. :smile:

Thanks.

After approval from Lennart by PM, I deleted the discussion about k's with more than one prime. I was unnecessarily verbose. There was no clear setup related problem nor problem related to PRPnet. It was simply related to splitting things up over multiple machines due to the large volume of k/n pairs that need to be tested in each group.

The edited posts show the # of k's with primes and k's remaining for each group so that is sufficient for administrative purposes.

I did want to post one request that was part of the discussion:

If everyone could kindly check his primes file before posting it on any base for more than one prime on a k, that would really help speed up the balancing of # of k's with primes and k's remaining on the various bases, especially bases with large conjectures like S63 here. If there is more than one prime for a k, please remove the higher prime(s). Also helpful on these large-conjectured bases would be to sort the primes by k-value before posting. For smaller-conjectured bases, sorting by n-value is perfectly fine since that is usually how people prefer to search higher n-ranges.


Thanks,
Gary

[QUOTE=gd_barnes;215368]I'd like to ask you Mark: After you have finished sieving, how are you running your ranges here? I'm assuming that 5M+ k/n pairs are too many for a single PRPnet server. Are you using multiple PRPnet servers running concerruntely or are you reloading the same server and running them in consecutive succession a 1000 or so k's at a time? Or can a single server handle such a huge load or are you doing something completely different? So far, I've only run clients on the 3.2.5/6 servers. I haven't attempted to load one with pairs yet.

I guess the above leads to another question: How many pairs can a PRPnet server handle at one time? Is that related to the memory of the machine or is there a server memory or pairs limit?[/QUOTE]

I am not using PRPNet for S63. The vast majority of tests take less then a second and the overhead of communication between client(s) and server (including database updates) would more then double the time needed to do a range. If you take a look at the first post you will see that I recommend using PFGW with the number_primes option.

My recommendation for those wanting to run one group on multiple cores is to split the file into multiple files, e.g. 2 cores means two files with 5000 k each. Sieve each file independently with srsieve (to 1e10) and use the -w option to create a .pfgw file. Change the first line of the .pfgw file to set up number_primes. Run each .pfgw file through independent instances of PFGW.

At home I have loaded a PRPNet server with over 500,000 pairs (R928/S928) and it handled that load without any problems. There are no hard-coded limits in the server WRT number of candidates. There is a setting (in 3.3.0) that limits the number of concurrent clients. I made that change because I have some anecdotal evidence that many hundreds of concurrent clients can crash the server. My recommendation has always been to steer clients towards getting enough work for at least one hour before communicating with the server because that helps to reduce the load on the server. There is no reason why thousands of clients couldn't work on the same server, as long they don't all try to get work at the same time.

OK, thanks Mark. I had forgotten about your suggestion in the 1st post. The funny thing is, the way you suggested is almost exactly the way I run a large majority of my new bases here if I feel I need no more than 2-3 cores for a base. For anything more than 3 cores, then I've had Max load them into a personal PRPnet or LLRnet (if running NPLB) server. Usually that's for high n-ranges and few k's.

I do wonder, though, if it would be worth it to go with your suggestion for, perhaps, n=1K-5K, remove all of the k's with primes in the sieve files, and then load the remainder in a PRPnet server. Nah, probably still too much manual intervention needed and overhead from the server. Tests at n=5K are pretty fast too. It's best to divide it up in 1000k to 2500k chunks and let it rip on a quad or 2 running good ole PFGW with the stop-on-prime option until all cores hit n=10K.

That's cool you were able to load 500K pairs into a server. It's nice to know that it can handle that.

I wouldn't load a PRPNet server until the PRP tests take at least 30 seconds each. The communication overhead adds too much relative time for faster tests.

@Gary:

Glad to see that there is only 3 missed k's in the entire range. Good job on completing the doublecheck.

@The rest of you: Good job on completing more and more of this drive. Do you think you can have it complete before the end of June? Anyone who feels like taking it higher? :smile:

Regards

Kenneth

[QUOTE=KEP;215629]@The rest of you: Good job on completing more and more of this drive. Do you think you can have it complete before the end of June? Anyone who feels like taking it higher? :smile:[/QUOTE]

No and not me. I would be surprised if it were done by the end of the year. At two months a range and 16 ranges, that is 32 months. So far only three individuals have shown interest in helping this drive.

Reserving Group 15

[quote=rogue;215634]No and not me. I would be surprised if it were done by the end of the year. At two months a range and 16 ranges, that is 32 months. So far only three individuals have shown interest in helping this drive.[/quote]

I'm not so pessimistic on it. I'll probably do one range at some point. I'd say we'll be done by the end of Sept.

Where do you get 2 months per range? Surely it doesn't take 2 months for a full quad to do a range. That is what I would put on it.

[quote=gd_barnes;215666]Where do you get 2 months per range? Surely it doesn't take 2 months for a full quad to do a range. That is what I would put on it.[/quote]
2 CPU months, (as he calculated previously and mentions in the first post) not 2 quad-CPU months. So if one core of a C2D 2.4 GHz was put on this, and nothing else, it'd probably take about 32 months. If 32 cores were put on it, you could finish it in a month. :smile: More likely is somewhere between these two extremes. Like 2-5 quads (equivalent average) finishing it in 4 to 1.6 months.

[quote=Mini-Geek;215670]2 CPU months, (as he calculated previously and mentions in the first post) not 2 quad-CPU months. So if one core of a C2D 2.4 GHz was put on this, and nothing else, it'd probably take about 32 months. If 32 cores were put on it, you could finish it in a month. :smile: More likely is somewhere between these two extremes. Like 2-5 quads (equivalent average) finishing it in 4 to 1.6 months.[/quote]

Ugh. I should learn to remember what I read. Thanks.

OK, if we have an average of 2 quads running this effort, like you said, that'd be 4 months or a Sept. completion date, close to what I was thinking. But a 1 quad average would put it at 8 months or Jan. of next year, close to what Mark said. We'll have to see what the interest is.

Seeing 17 and 19 everywhere, need some variation.. Gimme group 8 :smile:

Status update
 

Group 1 is past n=5K

6426 primes found so far

Attached are the results

Reserving group 9

Lennart

Reserving
[URL="http://home.roadrunner.com/%7Emrodenkirch/group16.zip"]Group 16[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group17.zip"]Group 17[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group18.zip"]Group 18[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group19.zip"]Group 19[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group20.zip"]Group 20[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group21.zip"]Group 21[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group22.zip"]Group 22[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group23.zip"]Group 23[/URL]

Lennart

[QUOTE=Lennart;216553]Reserving
[URL="http://home.roadrunner.com/%7Emrodenkirch/group16.zip"]Group 16[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group17.zip"]Group 17[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group18.zip"]Group 18[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group19.zip"]Group 19[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group20.zip"]Group 20[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group21.zip"]Group 21[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group22.zip"]Group 22[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group23.zip"]Group 23[/URL]

Lennart[/QUOTE]
what the...?? And you have reservations in other projects, too. Nice one.
Would you please tell me, how many machines / cores you have? :smile:

[quote=Xentar;216572]what the...?? And you have reservations in other projects, too. Nice one.
Would you please tell me, how many machines / cores you have? :smile:[/quote]


~ 150 core

Lennart

Group 07 results
 

7544 primes found

Taking group 10.

There are only four groups left. Get 'em while they're hot!

Status update
 

S63 group 1 is to n=9.5K

~80k tests left, hopefully it will be complete next week.

Group 1 is complete to n=10K

7556 primes found

attached are the primes.

The .out file for prime testing is too large to be included. I will email it to Gary.

[QUOTE=Mathew Steine;224879]Group 1 is complete to n=10K

7556 primes found

attached are the primes.

The .out file for prime testing is too large to be included. I will email it to Gary.[/QUOTE]

Could you please attach a file with a list of the k for which a prime was not found?

Sure thing

Group 11
 

I'd like to reserve group 11

[QUOTE=Lennart;216553]Reserving
[URL="http://home.roadrunner.com/%7Emrodenkirch/group16.zip"]Group 16[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group17.zip"]Group 17[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group18.zip"]Group 18[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group19.zip"]Group 19[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group20.zip"]Group 20[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group21.zip"]Group 21[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group22.zip"]Group 22[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group23.zip"]Group 23[/URL]

Lennart[/QUOTE]

It has been three months. Do you have an update?

[quote=rogue;225383]It has been three months. Do you have an update?[/quote]

G09 ans 16 is done to 90%

I had to restart all because a HD crach.

I can release
[URL="http://home.roadrunner.com/%7Emrodenkirch/group20.zip"]Group 20[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group21.zip"]Group 21[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group22.zip"]Group 22[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group23.zip"]Group 23[/URL]

Lennart

[quote=Lennart;225441]G09 and 16 is done to 90%

I had to restart all because a HD crash.

I can release
[URL="http://home.roadrunner.com/%7Emrodenkirch/group20.zip"]Group 20[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group21.zip"]Group 21[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group22.zip"]Group 22[/URL]
[URL="http://home.roadrunner.com/%7Emrodenkirch/group23.zip"]Group 23[/URL]

Lennart[/quote]

OK, to clarify: Do you [I]want [/I]to release them or are you saying that you [I]can[/I] release them?

It's a small point but we're in no huge hurry here so if you would like to continue by starting over on them, then that's no problem.

Sorry to hear about the HD crash.

[quote=gd_barnes;225470]OK, to clarify: Do you [I]want [/I]to release them or are you saying that you [I]can[/I] release them?

It's a small point but we're in no huge hurry here so if you would like to continue by starting over on them, then that's no problem.

Sorry to hear about the HD crash.[/quote]

It is betere to release them If the they are left when I have the other done I can reserve them again

Lennart

Group 8 resumed after some weeks summer break.
It is at n = 7000 at the moment.

Here are the prime found in G09

Edit:
There are 234 primes for duplicated k's so there are 7616 k's with primes and 2384 k's remaining for this group.


Lennart

[QUOTE=kar_bon;215639]Reserving Group 15[/QUOTE]

Do you have a status update? It has been more than three months.

[QUOTE=rogue;227941]Do you have a status update? It has been more than three months.[/QUOTE]

At n=4800 so far.

[QUOTE=kar_bon;227947]At n=4800 so far.[/QUOTE]

Ouch! Either you haven't worked on it full time, haven't sieved deeply enough, or are running it on a slow computer.

[QUOTE=rogue;227979]Ouch! Either you haven't worked on it full time, haven't sieved deeply enough, or are running it on a slow computer.[/QUOTE]

First one! Will do more the next weeks.

Group 10 Results
 

7607 primes found.

[quote=Lennart;227270]Here are the prime found in G09

Lennart[/quote]

Ian,

It would be virtually impossible for there to be only 2150 k's remaining for this range. Lennart does not remove duplicated primes for k's. After removing duplicates, there are 7616 unique k's with primes. Therefore there are 2384 k's remaining for the range.


Gary

[QUOTE]Lennart does not remove duplicated primes for k's.[/QUOTE]

Didn't know that, noted.

I am sorry that I uploaded the wrong file.

Lennart

I'm showing k's remaining on the pages for the 1st time for this drive; but only the contiguous ranges starting from the beginning, which means groups 1 thru 7 or k<=11680750. Stats for this range:

70,000 k's remaining at n=1K.
53,232 primes found for n=1K-10K.
16,768 k's remaining at n=10K.
76.05% k's eliminated; 23.95% remain.

Extrapolating to estimate k's remaining for the entire base at n=10K:
237,036 * 23.95% = 56,780

Xentar, as soon as you have group 8 done, we'll be complete for all ranges up through group 10 or k<=16440570. No pressure. :smile:

[QUOTE=gd_barnes;230254]Xentar, as soon as you have group 8 done, we'll be complete for all ranges up through group 10 or k<=16440570. No pressure. :smile:[/QUOTE]
It's at n=9600, so should be done next week.
After this, I will check the FFT sizes with PFGW 3.3.6 to be sure, no prime is missing.

Ok, group 8 is finished.
I tried to check the FFT sizes. I used PFGW 3.3.5 and 3.3.6, with the -F parameter to store the sizes in a text file. Then I compared both files (actually, I wrote a small python script for all this), and they are exactly the same, so there is no FFT difference.
Could this be possible, or should I check my script?

If this can be correct, I will post the results.

Attached you find the file with the primes.

Gary, do you need any other files? PFG.out? :smile: It's a 54MB zip file..

[QUOTE=Xentar;231417]Ok, group 8 is finished.
I tried to check the FFT sizes. I used PFGW 3.3.5 and 3.3.6, with the -F parameter to store the sizes in a text file. Then I compared both files (actually, I wrote a small python script for all this), and they are exactly the same, so there is no FFT difference.
Could this be possible, or should I check my script?

If this can be correct, I will post the results.[/QUOTE]

Mark, can you answer this?

Although it seems unlikely, I suppose that it would be possible that there are no FFT differences for 10,000 k's for n=1K-10K.

[QUOTE=Xentar;231515]Attached you find the file with the primes.

Gary, do you need any other files? PFG.out? :smile: It's a 54MB zip file..[/QUOTE]

Thanks for the primes. I typically ask for k's remaining but I can figure that out easily enough.

No, no results (.out) file needed for n<=25K on any huge-conjectured base (ck>~100K) like this. It's much too cumbersome.

[QUOTE=gd_barnes;232298]Mark, can you answer this?

Although it seems unlikely, I suppose that it would be possible that there are no FFT differences for 10,000 k's for n=1K-10K.[/QUOTE]

It is possible, but if you recall I had incorrectly linked one of those versions of PFGW and had to re-release. I suggest that you use -F for 9238*619^619-1 on both versions. If the FFT sizes are the same, then one of your PFGW's is wrong. If they are different, then I would say that everything is good.

[QUOTE=cmd]pfgw335.exe -F -q"9238*619^619-1"
PFGW Version 3.3.5.20100910.Win_Stable [GWNUM 25.14]
Special modular reduction using FFT length 768 on 9238*619^619-1

pfgw336.exe -F -q"9238*619^619-1"
PFGW Version 3.3.6.20100908.Win_Stable [GWNUM 25.14]
Special modular reduction using zero-padded FFT length 896 on 9238*619^619-1
[/QUOTE]
This seems to be correct.

But what I don't understand. After rechecking my input file sr_63.pfgw, even 3.3.5 uses "zero-padded FFT".

[QUOTE=Xentar;232354]This seems to be correct.

But what I don't understand. After rechecking my input file sr_63.pfgw, even 3.3.5 uses "zero-padded FFT".[/QUOTE]

Then the 3.3.5 you had d/l'd was the wrong one. You need to d/l it again.

To save you from doing it again, you can zip up and send me your sieved file of remaining k and I'll run it through a program I wrote that will output the ones that need to be retested. If you no longer have the file, I'll sieve the remaining k to 1e7 or 1e8 and do it.

[QUOTE=kar_bon;227981]First one! Will do more the next weeks.[/QUOTE]

Do you have an update?

Lennart, what is your status on groups 16-19?

[QUOTE=rogue;233879]Lennart, what is your status on groups 16-19?[/QUOTE]

I stoped all when the bug in pfgw,LLR (gwnum) come.

I think I have 16 done but right now i will not start 17-19.

To many subproject in the air now.

so anyone who will do 17-19 feel free to do that.

Lennart

[QUOTE=Lennart;233893]I stoped all when the bug in pfgw,LLR (gwnum) come.

I think I have 16 done but right now i will not start 17-19.

To many subproject in the air now.

so anyone who will do 17-19 feel free to do that.

Lennart[/QUOTE]

OK. Just post the results for group 16.

Note that I have done retesting for all k for this base for all n < 1000 and found no problems.

Taking groups 20 through 23. I added an option to phrot so that it can do something akin to the number_primes option in pfgw, thus my PPC can handle these ranges now that it is done with what it was working on.

[QUOTE=rogue;233896]OK. Just post the results for group 16.

Note that I have done retesting for all k for this base for all n < 1000 and found no problems.[/QUOTE]

Lennart,

Do you have the primes for group 16 completed now? If so, could you post them? Thanks.


Gary

G16 7641 primes

[QUOTE=kar_bon;227981]First one! Will do more the next weeks.[/QUOTE]

Karsten,

Can you provide a status update? It's been over 5 months now.


Gary

[QUOTE=gd_barnes;235281]Karsten,

Can you provide a status update? It's been over 5 months now.[/QUOTE]

After the rally I'll run this with more power now.
Currently 3680 k-values left at n=4900.

With full power I think it's ready in about 1-2 weeks.

Reserving group 12. I won't be going full-power on this until my S40 work is done, (rough estimate 1 week) but I started some sieving on it and thought I'd go for it. :smile:

Taking groups 13 and 14

Taking the last groups: 17, 18, and 19. If all goes well, this drive should complete by the end of January and thus can remove the sticky at that time.

appeldorff, do you have an update on your range?

I expect my range to complete in december or january. I was hoping to put my Phenom 965 on it but it seems the cooling cant keep up with more than 1 core. I'm thinking of getting a water cooler around christmas and if I get that the speed should increase by quite a bit.

So far I have a little over 400k tests to go

Group 12 complete. 7563 primes attached, 2437 k's remaining (should I include this file too?).

[QUOTE=Mini-Geek;240015]Group 12 complete. 7563 primes attached, 2437 k's remaining (should I include this file too?).[/QUOTE]

Yes, it makes it easier to put the list of remaining k together.