How to use sr2sieve
 

Hello,
can anyone tell me how to use sr2sieve to sieve riesel primes?
Many people tell it's so fast so I want to use it.
But the program needs a riesel.dat file and I couldn't make such one.

nuggetprime

If you want to help out the Riesel Sieve project,(apologies if I'm wrong) your best bet is to download BOINC and attach to boinc.rieselsieve.com .

They've just recently begun using sr2sieve in their sieving efforts, so you'll get what you want with only a little bit of hassle.

[I]nuggetprime[/I] what do you have as an input? Several NewPGen files? Single NewPGen file?
If you have a single NewPGen file then sr1sieve will work directly with it.
If you have several NewPGen files (you are sieving several k simultaneously) then you have to convert them first to ABCD format using srfile utility (part of srsieve package) and name it sr2data.txt.
Then you create file sr2work.txt, which will have two numbers per each line separated with comma, e.g.:[code]1000,2000[/code]Above would mean that sr2sieve will sieve sr2data.txt from p=1000*10^9 till 2000*10^9.
For more info consult help files of srfile, srsieve, sr1sieve, sr2sieve and sr5sieve - information is a little bit scattered :wink:

Note that with recent versions of sr2sieve can use the -i, -p, -P options in the same way as sr1sieve, so you don't have to create sr2work.txt.

(The -r and -s switches are just needed to interpret the .dat files, because there is no information in the file to distinguish Riesel from Proth numbers. If you use an ABCD format file then -r and -s are not needed.)

Thank you!
sr1sieve and sr2sieve are now working.
Is there a speed change between NewPGen and them?

nuggetprime

It depends on k you are sieving, but improvements in my case range from ~15% to double or almost tripple the NewPGen speed (the later achieved using "k8" executables under linux).

nugget:
With P4 versions (fastest that run under Windows, on Athlon64 or P4), the speedup while sieving a single k with sr1sieve ranges from 40% to 90% for most k. An additional 60% or so is available if you run a 64-bit version of linux (I installed Suse64 for this reason on a Conroe).

sr2sieve is slower on a single k than sr1, but efficiency rises roughly with the square root of number of k's sieved at once with sr2. In other words, sieving 4 k's at once will be twice as fast as sieving each of the 4 individually with sr1. This is a rough rule of thumb, and again varies with choice of k.

Also, sr2sieve sieves the entire range of n for all k's in the sieve-- you would get no speedup from sieving k1 from 300,000 to 600,000 and k2 from 600,000 to 1M, because their ranges do not overlap (in fact, it would be VERY slow, because it would really be sieving both from 300,000 to 1M, even though candidates are not there over the whole range). This is a byproduct of the method used to sieve multiple k's.

-Curtis

[QUOTE=geoff;100495]Note that with recent versions of sr2sieve can use the -i, -p, -P options in the same way as sr1sieve, so you don't have to create sr2work.txt.

(The -r and -s switches are just needed to interpret the .dat files, because there is no information in the file to distinguish Riesel from Proth numbers. If you use an ABCD format file then -r and -s are not needed.)[/QUOTE]

In one of your previous postings, [URL="http://www.mersenneforum.org/showpost.php?p=98849&postcount=67"]http://www.mersenneforum.org/showpost.php?p=98849&postcount=67[/URL]
You mentioned that srsieve is faster up to 32 bit? Is this still true?

Based off your instructions, I created a little bat file which automates the whole process, this should help some people getting into the RPS.

[CODE]srsieve -s 10 -v -a -P 4e9 -n 290e3 -N 500e3 "AAA*2^n-1" "BBB*2^n-1"
ren sr_2.abcd sr2data.txt
sr2sieve -v
ren sr2data.txt sr_2.abcd
srfile -a -k factors4.txt sr_2.abcd
srfile -G sr_2.abcd
[/CODE]

Simply save it as a .bat file and modify the first line with your own parameters.

Regards
Patrick

[QUOTE=Patrick123;100744]You mentioned that srsieve is faster up to 32 bit? Is this still true?
[/quote]
On 32-bit machines srsieve uses 32-bit arithmetic for sieving below 2^32 and then switches to 64-bit arithmetic above 2^32. sr1sieve and sr2sieve use 64-bit arithmetic for everything. Whether the faster arithmetic of srsieve below 2^32 overcomes the better algorithm of sr2sieve is hard to guess, but sieving up to 2^32 takes only a small fraction of the total sieve time in most cases anyway.

[quote]
ren sr_2.abcd sr2data.txt
sr2sieve -v
ren sr2data.txt sr_2.abcd
[/QUOTE]
You can simplify this a bit with newer sr2sieve versions: remove the ren statements and add `-i sr_2.abcd' to the sr2sieve parameters.

On k=19217385 NewPGen is a litte bit faster than sr1sieve!
sr1sieve rate ~ 1500k per sec.
NewPGen rate ~ 2000k per sec.

I'm running windows and I have a 3.2 GHZ P4.
I'm running sr1sieve for P4.

nuggetprime

[QUOTE=nuggetprime;100883]sr1sieve rate ~ 1500k per sec.
NewPGen rate ~ 2000k per sec.[/QUOTE]That is odd, but then again, P4 is not best suited for factoring / sieving jobs :wink:
BTW: did you use newest executables (1.0.14)? Try with both AMD and Intel versions... Also try sieving complete range of "p" and measure the total time it takes NewPGen and sr1sieve to complete it.

sr1sieve
 

The sr1sieve for AMD is about 100k faster than the sr1sieve for Intel but nepgen is still faster(over 2000k per sec.)
Which Intel processor is the best for primality proving and sieving?
Probably I will buy one in a few years. Now I'm still using my 3.2 GHZ Intel P4:smile: .

nuggetprime

[QUOTE]Which Intel processor is the best for primality proving and sieving?[/QUOTE]

Core2 duo/quad

Do you know something to make testing faster, or is 6 months time the best what I can get for n=300k-500k?

I am wondering the same. Sieved up to 300Billion from 200k-500k and there are a bit over 30k candidates left :/

Guys, you are using the best number-crunching software in the world. Sieving and then LLR is the best approach. Tricks like running sieves on AMD and LLR on P4 are all I can suggest. WRT core2d/q: wait a few months and they will be very cheap because AMD is going to release its new chip... :wink:

Well, yeah, but still :P

Anyway, school holiday wioll start from tomorrow, and last for a week. I talked to my computer class teacher, and she allowed me to use LLR on as many computers as I want(They have ~25 Pentium 4's with 2.4Ghz of raw power:flex: ) So I hope to find some primes in a few days :)

[QUOTE=paulunderwood;100890]Core2 duo/quad[/QUOTE]IMO: For C2Q you should not exceed FFT size of 112-128kB - above that you will get a large performance hit per core.

Oops, wrong thread.

kuratkull-- it's hard to oversieve in general. Spending one CPU week sieving when LLR on the range will take six months is not too long. On P4-2.4, LLR at 400k is around 6 minutes (depends on k value). When your sieve removes a candidate every 6 minutes, the sieve is deep enough. Note that from 200k to 500k, the average candidate is closer to 400k than the actual midpoint, because time to LLR does not increase linearly (it goes up as n^2, so 500k is about 6x longer than 200k per test). You can likely sieve to 1T or so using sr1sieve before splitting the file onto so many machines for LLR work.
Also, note with even a dozen computers, 6 CPU months of work will take 15 days per machine. When you choose your next k-value, set up the sieve to run to 600k or 700k (or 1M!), so you don't have to distribute and collect the files after just 2 weeks. But that's in the future.. for now, enjoy the results your labor of installing LLR on school computers provides!
-Curtis

Since you have only one week it's the best to go with pre-sieved candidates. For example our 5th Drive. I also have 8 Ks we got from Masser in the 700k-1M range, if you are interested let me know.

But before starting mass production please test LLR on a single machine using known primes to confirm that everything is all right.

Thanks for your comments :)

Yes, I have done testing before, and everything works, primes are found and such.

I don't have ONLY one week, I just asked my teacher if she would allow me, and she said that holiday is coming, and that would be a good opportunity, and since I get along with her pretty well, I don't think that continuing the LLR'ing after the holiday will be a problem.
I only got to sieve until 270Bil :/ (I didn't have enough time, because I started to sieve a few days ago, but talked to her about using the school computers just yesterday).

But now I'm on holiday and my school computers are doing some hard work :flex:

Hoping to find my FIRST prime soon :)

Well, 25 P4s should be quite a help to find your first prime :shock:

I found out that Newpgen is much faster with fixed n than with fixed k. On fixed n I get speeds around 0.1 billion per second.

[QUOTE=nuggetprime;100883]On k=19217385 NewPGen is a litte bit faster than sr1sieve!
sr1sieve rate ~ 1500k per sec.
NewPGen rate ~ 2000k per sec.

I'm running windows and I have a 3.2 GHZ P4.
I'm running sr1sieve for P4.
[/QUOTE]

It is always a good idea to test all the options before spending a lot of CPU time sieving. sr1sieve uses a different algorithm to NewPGen and there will be cases when one is faster then the other.

However I timed your sequence over the range 100,000 < n < 3,000,000, 20000e8 < p < 20001e8 on a 2.8GHz P4/Celeron (Linux) and got these results:
NewPGen 2.82: 231 sec.
sr1sieve-intel 1.0.15: 165 sec.

Can you tell me what range of n you were sieving, and at what p? If p is too small and there are many factors being found per second, then sr1sieve often spends more time checking the factors and printing them to the screen than it spends actually finding them.

[QUOTE=geoff;101104]If p is too small and there are many factors being found per second, then sr1sieve often spends more time checking the factors and printing them to the screen than it spends actually finding them.[/QUOTE]I don't know if it would be of any use but maybe "verbose=0" switch could be implemented, especially for benchmarking? Also I would welcome a Windows (2000/XP) service :bow:

[QUOTE=Cruelty;101183]I don't know if it would be of any use but maybe "verbose=0" switch could be implemented, especially for benchmarking?
[/quote]
I may add a --quiet switch to a future version. For benchmarking the best approach is simply to use a high value for --pmin. If you expect to sieve to about p=X, then benchmarking at about p=X/2 should be ideal.

It is still necessary to sieve enough of the small factors out (so that the character of the sequence shows up in the terms left in the sieve) before starting benchmarking, but sieving to p=10 million or so should be sufficient in most cases, this usually only takes a minute with NewPGen or srsieve.

[quote] Also I would welcome a Windows (2000/XP) service :bow:[/QUOTE]

I guess a Windows service is like a daemon in unix? I compile the windows binaries using the mingw32 cross-compiler from Linux, it only has a small subset of the normal library functions to work with, signal handling in particular is almost non-existent. It might be possible using cygwin, but I am still looking for a cygwin cross-compiler.

p from 200 billion to 500 billion,
n = 300k-500k

[QUOTE=nuggetprime;101537]p from 200 billion to 500 billion,
n = 300k-500k[/QUOTE]

You are right, NewPGen is faster over that range. I think it is mainly the short range of n that makes the difference, more than the particular value for k.

practical limits on number of k's in sieve file
 

Hello geoff,

I am interested in performing multiple k sieves at once. I understand that sr2sieve would be the best choice of program.

What are the practical limits on the number of k's in a single sieve file, from a performance standpoint? I would like to have as many k's as possible -- each k would have approximately 10 to 100 n values associated with it. How does the performance scale as I increase the number of k's?

Thank you, in advance,

-- monst

I think once you get enough k's in the sieve file, srsieve becomes more efficient than sr2sieve.

Of course, if you're starting the sieving from scratch, you're going to need to use srsieve or NewPGen to start it out anyway (since sr2sieve won't sieve below a certain level of p, not to mention that it can't create a new sieve file, only sieve existing ones).

sieving suggestions...
 

Monst,

Below are cut-and-pastes of some relevant parts of some posts from another thread that explain quite a bit in detail. Bottom line:
1. For 1-2 k's, use sr1sieve. Running it twice is still usually faster than running any other program once with 2 k's.
2. For 3 to about 50 k's, use sr2sieve.
3. For about 50 k's to (I think) upwards of 100,000 k's, use srsieve.
4. For fixed-n searches and other kinds of 'specialty' searches like for twins, Sophie-Germains, Cunningham Chains, etc., use NewPGen.

In other words, the limit is very high on the # of k's that you can sieve at once.

For the use of sr1sieve and sr2sieve, you'll have to use srsieve first to sieve to the limit required by those programs (generally the largest k that you are sieving) but I recommend that you sieve to at least P=100M first.

All of these programs smoke NewPgen on speed for fixed-k searches. NewPgen was more designed for fixed-n searches.



[quote=gd_barnes;113391]Just to clarify to make sure I understand you correctly...Sometime late on Monday, you will have completed searching the range of 100K < k < 200K for n < 7000. Is that correct?

Suggestion...Having searched across large ranges of k quite a bit myself, I have found srsieve to be much faster than sr2sieve for a very large # of k's like you are doing here. You might run both over the same range of p for a small test and see which one is faster on your machine. I think srsieve will be much faster for the 50,000 k's that you're doing here (assuming that I understand you correctly).

My experience...when doing the double-checking for k <= 1001 up to n=50K, srsieve was faster in all cases, even when I reduced the range of k to just 300 < k < 1001. But when I did my 'large heavy-weight' k sieve for only 12 k's from k=19437 up to k=3428677395 from n=200K to 400K, sr2sieve was always faster. It probably has to do with actual factor removal as the program progresses on srsieve vs. just writing factors out and removing them later on sr2sieve. I think a large # of k's makes immediate removal by the program much faster vs. removing them at a few points along the way or all at the end using srfile.


Gary[/quote]

[quote=gd_barnes;114091]Anon,
Based on my experience, here are my suggestions for sieving software in different situations:

1. 1 to 2 k's of the form k*2^n+/-1: use sr1sieve. It's fast enough that running it twice is usually faster than running anything else once with 2 k's.

2. 3 to about 50 k's: use sr2sieve.

3. About 50 k's or more: use srsieve.

4. Very specific types of searches like SG's or twins: use NewPGen.

The 50 k's is a very rough estimate. If you have anywhere between about 20 and 100 k's you want to sieve, I'd suggest testing both programs and see which is faster. If about the same, use srsieve because it doesn't require as many steps and so is a little easier to use.

Once you get used to the srsieve series of programs, you'll rarely go back to NewPGen for standard searches.

If you haven't used sr2sieve, it takes a little getting used to. There are a few hoops to jump though on it. Curtis is the best resource for getting all of the ins and outs of it.


Gary[/quote]

requests for additional functionality in srsieve
 

Gary,

Thank you for the advice. I have srsieve working on about 2400 sequences -- from your input it seems that srsieve is a better choice than sr2sieve.

Geoff,

I like srsieve a lot! I converted hundreds of NewPGen fixed-n sieve files with thousands of k's in them to work with srsieve and my boxes are humming along.

When working with srsieve and sr2sieve, there are two pieces of functionality that sr2sieve has that would really help me in srsieve.
1.) x86-64 functionality for Windows (I don't see a download available for this on your site)
2.) the -z and -Z command line options to adjust priority (with the default set to Don't adjust priority, I find myself going to the task manager to switch to low priority after every time I restart the sieve)

Let me know if I am missing something in build 0.6.7, if that functionality is already there.

Thanks,

-- monst

[quote=monst;119467]
2.) the -z and -Z command line options to adjust priority (with the default set to Don't adjust priority, I find myself going to the task manager to switch to low priority after every time I restart the sieve)

Let me know if I am missing something in build 0.6.7, if that functionality is already there.

Thanks,

-- monst[/quote]

Good request! When I do sieving with srsieve on 300-500 k's at once for some of my double-check efforts, I have the same issue. sr1sieve and sr2sieve default to low priority but srsieve defaults to medium priority and I have to go in to the task manager to adjust it to low.

Personally I wouldn't even need the -z or -Z options. If srsieve just defaulted to low priority, that would work for me but I'm sure others may want the option to adjust it at the command line.


Gary

[quote=gd_barnes;119473]
Personally I wouldn't even need the -z or -Z options. If srsieve just defaulted to low priority, that would work for me but I'm sure others may want the option to adjust it at the command line.
Gary[/quote]


That implementation, defaulting to low priority (or idle priority), would be fine with me too.

[QUOTE=monst;119297]Hello geoff,

I am interested in performing multiple k sieves at once. I understand that sr2sieve would be the best choice of program.

What are the practical limits on the number of k's in a single sieve file, from a performance standpoint? I would like to have as many k's as possible -- each k would have approximately 10 to 100 n values associated with it. How does the performance scale as I increase the number of k's?
[/QUOTE]

In principle, for a sieve with many (hundreds) of k and very large range (10 of millions) of n, the speed should be proportional to sqrt(number of k * range of n). But when there are many more k and only a few n this will not apply, and in this case you might be better off with a trial factoring algorithm.

sr2sieve is optimised for data sets of about the size of the main distributed projects it is being currently being used for, i.e. SR5 (200-300 k, 2 million n range), RS (66 k, 20 million n range), PSP (17 k, 50 million n range). srsieve might be faster than sr2sieve when the range of n is smaller.

If you are thinking of trying to use sr2sieve as a replacement for a (multiple) fixed-n sieve, don't bother, it will be slow.

[QUOTE=monst;119467]When working with srsieve and sr2sieve, there are two pieces of functionality that sr2sieve has that would really help me in srsieve.
1.) x86-64 functionality for Windows (I don't see a download available for this on your site)
2.) the -z and -Z command line options to adjust priority (with the default set to Don't adjust priority, I find myself going to the task manager to switch to low priority after every time I restart the sieve)
[/QUOTE]

I have uploaded srsieve 0.6.8 which has the -z and -Z switches, which work the same as in sr2sieve.

Building a x86-64 Windows executable is on my TODO list, but it will take some time as a lot of assembly code needs to be translated to WIN64 calling conventions, it is not just a matter of changing compiler options.

All new development happens in sr2sieve. A lot of it gets into sr1sieve fairly quickly because most of the important code is shared between sr1sieve and sr2sieve. However there is now very little common code between srsieve and sr2sieve, and so updating srsieve is becoming difficult.

More questions about sr(n)sieve...
 

Hello Geoff,

Thanks for the new build of srsieve. I have downloaded it and have it running.

I will stay with NewPGen and perform multiple fixed-n sieving when the range of n is small.

I have several projects that have extended n-ranges and would be more suited to srsieve, sr1sieve, or sr2sieve. However, each sequence has a different base b. Do you recommend (performance-wise) running multiple sequences when the bases are different? (I noticed that on the prime pages, it stated that sr2sieve is for base 2, but I read a post that you enhanced it and lifted this restriction.) If so, would you run srsieve or sr2sieve? If not, I'll stay with sr1sieve.

With regards to 64-bits, I appreciate your comments on the different code bases: sr(n)sieve vs. srsieve. How much does 64-bit improve sieving performance on Windows? (I know with Prime95, there is a huge benefit when trial factoring, but not as much when LL testing.) This could sway me to move to sr2sieve with fewer sequences since it already has an x86-64 version.

Finally, where is multi-threading on your TODO list? -- Just curious...

Thanks, in advance.

[QUOTE=monst;119788]Do you recommend (performance-wise) running multiple sequences when the bases are different?[/quote]
sr2sieve doesn't support multiple bases in a single run, so you'll have to split your runs based on the bases. Also, initial sieving should be performed by srsieve (the details are explained in the README file coming with sr2sieve). Other than that, go nuts!

[QUOTE=monst;119788]How much does 64-bit improve sieving performance on Windows?[/quote]
I've seen people posting nearly 2x speed compared to 32-bit client (actually almost 80% improvement). YMMV.

[QUOTE=monst;119788]Finally, where is multi-threading on your TODO list?[/QUOTE]
I'll leave this one up to geoff :smile:

Thanks. I've got sr1sieve running in batch mode and I'll move it to a 64-bit machine tomorrow.

monst
 

If it's not a secret, what kind of projects are you working on?

I'm not sure we can give you the best advice based on sparse information you provided so far.