Riesel base 3 reservations/statuses/primes

[rogue]

Originally Posted by KEP

Great job Mark. It appears that 1 of my cores will free up on august 7th, so from around that date, if you need some ressources to test your changes, I can lend you a hand ... I hope though, that it is fully implemented and tested by then, so I can start up new ranges for R3

The results so far have been disappointing and I don't know why. I need to debug to figure out where it is wasting so much time.

[VBCurtis]

Originally Posted by VBCurtis

Great, thanks! I only got to 1e9 anyway, part-time office machine.

I'll begin 4-5G, from 100k to 500k. Since there is so much already sieved, I'll let it run at the office for a while.

I finally have a full-time core on this sieve. I'll have 100-200k ready for Reb/boinc roughly 1 Aug.
Sorry for the delay.

[rogue]

I'm actually making progress on this.


1) Run fixed-n sieve (newpgen) from n=1 to n=20 (maybe higher)
2) Write a custom program to:
a) Take files from step 1 along with the base, mink, maxk, and c and produce pl_MOB.txt and pl_prime.txt.
b) For remaining k, sieve (srsieve) and test (pfgw) ranges in chunks of n. New primes are added to pl_prime.txt
c) When n=25000 is reached, output pl_remain.txt


I have not done 2a, but I don't think that will be too hard if I understand the logic correctly. For 2b, it reads in the files created from newpgen to build a list of remaining k. It then takes those k thru multiple iterations using a table that consists of: min n, max n, max p, # of k. It sieves the k in chunks from min n to max n up to max p. It then runs those through srsieve and pfgw. As is progresses, it builds pl_prime.txt and at the end outputs pl_remain.txt.

[rogue]

Originally Posted by rogue

The results so far have been disappointing and I don't know why. I need to debug to figure out where it is wasting so much time.

I introduced a massive memory leak with my change. I need to track that down if I'm going to continue pursuing it.

[rogue]

Originally Posted by rogue

I'm actually making progress on this.


1) Run fixed-n sieve (newpgen) from n=1 to n=20 (maybe higher)
2) Write a custom program to:
a) Take files from step 1 along with the base, mink, maxk, and c and produce pl_MOB.txt and pl_prime.txt.
b) For remaining k, sieve (srsieve) and test (pfgw) ranges in chunks of n. New primes are added to pl_prime.txt
c) When n=25000 is reached, output pl_remain.txt


I have not done 2a, but I don't think that will be too hard if I understand the logic correctly. For 2b, it reads in the files created from newpgen to build a list of remaining k. It then takes those k thru multiple iterations using a table that consists of: min n, max n, max p, # of k. It sieves the k in chunks from min n to max n up to max p. It then runs those through srsieve and pfgw. As is progresses, it builds pl_prime.txt and at the end outputs pl_remain.txt.

Coding done, now testing. With this setup, I estimate that a range of 5M can be completed in under 6 hours on my laptop. This means about 20M per day per core extrapolated to 12 days for a range of 1G using all four cores of my laptop. That crushes all other results. It breaks down like this:

1) Running newpgen from n=1 to n=20, about 15 minutes. I ran to n=24, but that takes much longer. I think that n=22 is a good number. Note that this time is for a range of 1G.
2) For MOB, I sieve from mink to maxk then using those results to eliminate MOB k from the starting range. This takes a couple of seconds (for a range of 5M).
3) Applying newpgen results takes about 10 seconds.
4) The sieving/prp testing is done in 6 steps with a different range of n at each step. This takes close to 5.5 hours.

I am running a test now. If it produces the correct output (and I think it will), I'll provide the results, my program, and instructions for using it. This program can be used for all conjectures, so when people eventually get to S3 or some of the other nasty conjectures, it can be used for those as well, although it will require some additional testing as I have focused solely on R3. The one thing it doesn't do is the GFN checks that are in the script, but that should be fairly easy to do.

[KEP]

Great job Mark. It appears that your program is 2.5 times faster then the -f0 to n=2500 and sieve and prp test and prove approach

If this also is true for my hasswell, it means that I can do 45.625G per year on my Hasswell alone. Since my Sandy Bridge is about 2/3 the speed of my Hasswell, it means that for R3 I can do (in theory) 75.625G per year on 8 cores. Looking forward to be seeing what your new program is looking like and I hope that within the first 3-6 months after release, that testing can be done, to see if the program really works properly on all other bases as it you elaborate that it does on R3, because then I might just give the entire base 3 a swing

Again awesome work Mark, thanks for your many contributions to this project

[rogue]

I'm doing a final run, hopefully without bugs.

Gary, maybe all of the posts in this thread that are discussing the "fastest method" should be moved to another thread. In fact, if my program does work as well as I hope it to work, maybe it should replace the software/instructions/questions thread. That is you decision.

[gd_barnes]

Originally Posted by rogue

I'm doing a final run, hopefully without bugs.

Gary, maybe all of the posts in this thread that are discussing the "fastest method" should be moved to another thread. In fact, if my program does work as well as I hope it to work, maybe it should replace the software/instructions/questions thread. That is you decision.

I just got back from a week trip. This looks to be very impressive work.

I like your "testing new ranges for Riesel base 3" thread. I'll move everything related to testing and timings and software discussions about Riesel base 3 to that thread as soon I look at it a little more.

I think that the main important thing is that before anyone reserves any new ranges to be tested on your new software, it must be extensively parallel tested, that is closely compared, with previously completed ranges. All primes, remaining, and MOB must be the same. If everything comes out identical for perhaps a k=100M range then it should be good to go.

[lalera]

Originally Posted by gd_barnes

I just got back from a week trip. This looks to be very impressive work.

I like your "testing new ranges for Riesel base 3" thread. I'll move everything related to testing and timings and software discussions about Riesel base 3 to that thread as soon I look at it a little more.

I think that the main important thing is that before anyone reserves any new ranges to be tested on your new software, it must be extensively parallel tested, that is closely compared, with previously completed ranges. All primes, remaining, and MOB must be the same. If everything comes out identical for perhaps a k=100M range then it should be good to go.

which range ?

[rogue]

Originally Posted by gd_barnes

I just got back from a week trip. This looks to be very impressive work.

I like your "testing new ranges for Riesel base 3" thread. I'll move everything related to testing and timings and software discussions about Riesel base 3 to that thread as soon I look at it a little more.

I think that the main important thing is that before anyone reserves any new ranges to be tested on your new software, it must be extensively parallel tested, that is closely compared, with previously completed ranges. All primes, remaining, and MOB must be the same. If everything comes out identical for perhaps a k=100M range then it should be good to go.

I have compared pl_MOB and pl_remain for that range of 5M and there were no differences. I did not compare pl_prime.

[KEP]

Originally Posted by lalera

which range ?

You could test from:

k=20,000,000,002 to
k=21,000,000,000

using the starting base script.

I've just begun testing the same range using rogues srbsieve. That way, we can establish a great doublecheck and see if the program works as should. Also we could have another 1G range doublechecked, wich is not entirely a bad idea to do once in a while

Can Gary/you accept this or do you have other suggestions?