Bigger and better GPU sieving drive: k<10000 n<2M

[mdettweiler]

Hi all,

Okay, this is what the admins have been cooking up under wraps for the last week or so. Not long after we started our earlier drive for k<=1001, n=1M-2M, Lennart dropped me a line saying that we could actually sieve much, much more efficiently by making it for k<10000 instead of just k<=1001. (I'm kind of new to this ppsieve stuff, so I'm still getting a feel for how it scales. ) Anyway, to make a long story short, we eventually decided to include n<1M too (may as well get that to GPU-optimal depth while we're at it) and, due to the sheer size of the new effort, it's going to be a collaborative effort with PrimeGrid. As before, at the end of the sieve the file will be released for the benefit of the entire Riesel prime community, except that this time there will be no stipulation that primes found with the file be reported jointly with NPLB. (We decided to drop that because it would get totally crazy asking people to add NPLB and PrimeGrid to each of the primes found with this file. )

The plan is this: the sieve will be coordinated from two places, PrimeGrid's PrimeSearchTeam forum (where they do many of their behind-the-scenes sieves), and here at NPLB. The PST forum's reservation table will be the primary one, and I will reserve large p-ranges from there under the name "NPLB" and list them as available here. When those are completed, I'll combine the factors and send them back to Lennart for integration into the main table.

The PST reservation thread was started a few days ago and has already gotten p<10T reserved (not quite completed) for the n<1M portion of the range. The 1M-2M portion is in progress for that same range and on track to finish within a week. The reason for splitting it up like this is because the sieve file is enormous (gigabytes, IIRC) and there's little speed penalty to splitting it up anyway. The plan is to combine the ranges at p=10T and continue without a sieve file from there on up. Yes, you read that right. ppsieve can run without a sieve file with effectively no speed penalty! The only disadvantage is that the factor files are much bigger due to all the duplicate factors floating around, but factors thin out quickly enough that we should be OK going without a sieve file after 10T.

Again due to the sheer size of the sieve file, I'm not even going to bother opening reservations here until the sieve has passed 10T. So in the meantime, I'll point you guys to the PST reservation thread. Note that though you have to register for an account there in order to view the forum, one of the admins there should get you approved in less than 24 hours. BTW, they refer to this project as "RSP" (Riesel sieve project) over there; not to be confused with the old Riesel Sieve project doing the base 2 Riesel conjecture, or with the similarly-appreviated Riesel Prime Search (RPS). This is just the short identifier they're using for it over there like they have for all of their subprojects. Over here, it will merely be the k<10000, n<2M sieving drive.

Once the sieve has been completed to 10T, I will open reservations here and we'll continue to p=50T or so. At that point, PrimeGrid is planning to put it into BOINC and start really kicking some butt. Manual reservations through this thread should still be available even after the PrimeGrid half goes BOINC.
Edit (10/5): We've decided to go ahead and start sieving from 10T+ right away. The fact that not everything's done <10T doesn't make a difference since we're running without a sieve file.

Even farther down the line, the plan is to combine this with PrimeGrid's existing Proth Prime Search Extended (PPSE) sieve for the same k- and n-range. Ken has come up with a new version of ppsieve (actually it's based on tpsieve, but they're mostly the same underneath anyway) which can sieve +1 and -1 forms simultaneously with no speed penalty, so this will improve our efficiency even more.

Reservations table (for NPLB's portions of the sieve ONLY--see PST's thread for the canonical overall table):

P-range         reserved by     status       est. completion date
10000G-12000G   frmky           complete
12000G-15000G   vaughan         complete
15000G-17000G   frmky           complete
17000G-19000G   vaughan         complete
19000G-20000G   frmky           complete
20000G-30000G   PST forum
30000G-40000G   frmky           complete
40000G-50000G   PST forum
50000G-60000G   frmky           complete

NPLB's effort for n<2M is now complete and has been turned over to PrimeGrid's BOINC process, which is working on bringing the sieving up to P=500T. We are now working on assisting with the sieving for n=2M-3M. See this thread for more details.

Another post is coming soon with details on how to get started. (Update: now posted, see here.)

Max

[mdettweiler]

Hi all,

I've now confirmed with the PrimeGrid folks that it will be OK for us to start working on p=10T+ even as they finish up work <10T. (Since we're running without a sieve file for 10T+, there is no disadvantage to doing this a little "out of order".) Thanks henryzz for the suggestion on this!

To participate, follow these steps:

-Download the appropriate version of ppsieve for your computer. CPU and CUDA versions can be found here for Windows, Linux, and MacIntel. If you have an ATI GPU, go here and grab the OpenCL version.
Note: CUDA users will need the cudart-2.0 libraries in a place where ppsieve can see them. They are from a slightly older-than-latest version of the CUDA toolkit, so they probably didn't come with your installed CUDA toolkit. For Windows, download cudart.dll and plop it in your ppsieve folder. For Linux, download libcudart.so.2 for 32-bit or 64-bit and stick it in your CUDA libraries directory (probably /usr/local/cuda/lib/ or /usr/local/cuda/lib64/--at least that's where it is on Gary's machine).

-Extract ppsieve to its own folder somewhere.

-Reserve a range from this thread.

-Run the appropriate command line for your ppsieve version and range. Quoted from John at the PST forum:

A sample command line is below:

./ppsieve(version) -p##G -P##G -k 5 - K 9999 - N 2M -ffrsp2M_##G-##G.txt -t# -R -q
(ppsieve(version).exe for Windows)

(version) - to match what version you're using
-p##G - start range (pmin)
-P##G - stop range (pmax)
-k - kmin for sieve
-K - kmax for sieve
-N - nmax for sieve (-n nmin is not needed)
-f - frsp_##G-##G - to match your start/stop range; this is the factor file you'll be uploading
-t# - to match cores being used (if only single core, just delete) (GPU's do not use this)
-R - VERY IMPORTANT!!! This makes sure the sieve is run for -1 factors
-q - to run in quiet mode

For example, a 10000G-10400G range for a quad running 64 bit Linux CPU would be noted as follows:

BEFORE: ./ppsieve-x86_64-linux -p##G -P##G -k 5 -K 9999 -N 2M -ffrsp2M_####G-####G.txt -t# -R -q
AFTER: ./ppsieve-x86_64-linux -p10000G -P10400G -k 5 -K 9999 -N 2M -ffrsp2M_10000G-10400G.txt -t4 -R -q

NOTE: you can also use e9 instead of G.

-Put your factors in a zip file and upload them to http://pgllr.mine.nu/sieves/rsp/factors/. If you have problems uploading there (like my browser does--strangest thing), email them to me (max@noprimeleftbehind.net) and I'll get them where they need to go.

-Reserve another range and do it again.

Max

[mdettweiler]

Reserving 11T-12T for Gary's GPU.

[mdettweiler]

After some discussion with John, we've determined a naming convention for p>10T factor files:

frsp2M_##G-##G.txt

Apologies for the ambiguity on this at first--originally we had both forums' instructions saying that the files should be "frsp1M...", which is being used to denote the n=1M-2M sieve for p<10T currently in progress in the PST forum.

Greg, currently you're the only other person running a p>10T range; you may have already started your range with the "1M" factor file naming convention, in which case you'll want to rename it accordingly at the end. Again, apologies for any inconvenience this may cause.

[Mini-Geek]

BEFORE: ./ppsieve-x86_64-linux -p##G -P##G -k 5 -K 9999 -N 2M -ffrsp1M_####G-####G.txt -t# -R -q
AFTER: ./ppsieve-x86_64-linux -p10000G -P10400G -k 5 -K 9999 -N 2M -ffrsp1M_10000G-10400G.txt -t4 -R -q

This still has the old file name convention.

[mdettweiler]

Originally Posted by Mini-Geek

This still has the old file name convention.

Whoops, fixed.

[vaughan]

12000G - 13000G reserved by vaughan

[vaughan]

13000G - 14000G reserved by vaughan

[vaughan]

14000G - 15000G reserved by vaughan

[frmky]

10000G-11000G is done.
Sieve complete: 10000000000000 <= p < 11000000000000
Found 3436184 factors
count=33353349498,sum=0x7454bcfd2eb4b56e
Elapsed time: 109172.21 sec. (0.01 init + 109172.20 sieve) at 5105010 p/sec.

Reserving 15-16T.

[frmky]

15T-16T:
p=15066084667393, 5.190M p/sec, 0.05 CPU cores, 6.6% done. ETA 10 Oct 15:37

Also reserving 16T-17T frmky
p=16000587464705, 4.880M p/sec, 0.08 CPU cores, 0.1% done. ETA 10 Oct 22:09