Wilson-prime search practicalities

[Christenson]

Originally Posted by Jeff Gilchrist

I would like to help out with the search. You guys are taking R. Gerbicz's code and compiling it, but what command line parameters are you using?

I see talk about different ranges are better for things, but if you have core2 based CPUs what would the best settings be? Did I see that someone said it is better to run it on 1 core because running it on multiple ends up being the same speed?

Time to rename this thread...something like "Wil(l there be more )son(s of) primes search(ed for?)"

[MrRepunit]

Originally Posted by Jeff Gilchrist

I would like to help out with the search. You guys are taking R. Gerbicz's code and compiling it, but what command line parameters are you using?

My command line parameter are:
icc -O3 -lgmp wilsontest.c -o wilsontest.exe
(needs of course GMP installed properly)

If you mean the parameters for running it:
There are no real command line parameters as you have to interactivly type it in. But you can mimic the input with echo (assuming that you start from scratch)
echo -e "<start>\n<end>\n<interval>\n0\n" | ./wilsontest.exe

[Jeff Gilchrist]

Originally Posted by MrRepunit

My command line parameter are:
icc -O3 -lgmp wilsontest.c -o wilsontest.exe
(needs of course GMP installed properly)

No I meant command line parameters to *run* the program, not compile it. I now have a working 64bit Windows binary if anyone is interested.

I understand the start and end value part, but when it says how many primes to test what are people using? It is best to keep increasing that until you use as much RAM as possible to speed things up? And someone was saying it is better to use 1 core with lots of RAM instead of multiple cores with your RAM split between them?

I will try the range 7e10 to 8e10 then.

Jeff.

[MrRepunit]

Originally Posted by Jeff Gilchrist

No I meant command line parameters to *run* the program, not compile it. I now have a working 64bit Windows binary if anyone is interested.

I understand the start and end value part, but when it says how many primes to test what are people using? It is best to keep increasing that until you use as much RAM as possible to speed things up? And someone was saying it is better to use 1 core with lots of RAM instead of multiple cores with your RAM split between them?

I will try the range 7e10 to 8e10 then.

Jeff.

That what I was thinking after I answered, so I edited my last post before seeing yours.
Concerning the runtime with splitting and without:
I don't have exact timings, but my feeling is that using more cores should be faster. By looking at the timings of fivemack I see that a factor of ten in the used interval (4E6 to 4E7) just gives a speedup of a factor 2.5. So ten cores instead of one on that range would be faster.

[MrRepunit]

Originally Posted by Jeff Gilchrist

I now have a working 64bit Windows binary if anyone is interested.

I am interested. Is it on your usual webpage?

[fivemack]

I've split this into algorithm-discussion and practicalities threads; am happy to rearrange if it becomes unwieldy again

[fivemack]

Originally Posted by MrRepunit

I don't have exact timings, but my feeling is that using more cores should be faster. By looking at the timings of fivemack I see that a factor of ten in the used interval (4E6 to 4E7) just gives a speedup of a factor 2.5. So ten cores instead of one on that range would be faster.

I'm still not quite sure what the right answer is; it's going to depend on how much memory you have per core, and it is clear that the memory usage does depend on the value of 'interval' even if interval is much larger than the number of primes in the range.

At the moment, if you have about (maybe a little over) one gigabyte per core, I would run a 1e9 range on each core and use 2e7 as the interval parameter. But I haven't yet completed a calibration run with that parameter choice, I expect it to take a bit over 24 hours but I'm not at all sure how much over.

[Jeff Gilchrist]

Originally Posted by fivemack

I'm still not quite sure what the right answer is; it's going to depend on how much memory you have per core, and it is clear that the memory usage does depend on the value of 'interval' even if interval is much larger than the number of primes in the range.

At the moment, if you have about (maybe a little over) one gigabyte per core, I would run a 1e9 range on each core and use 2e7 as the interval parameter. But I haven't yet completed a calibration run with that parameter choice, I expect it to take a bit over 24 hours but I'm not at all sure how much over.

I have 8GB on my system, but I tried running the tool with the full 7e10 to 8e10 start and finish with a 20,000,000 interval and it seemed to be using about 1.2GB of RAM. Now I'm trying just 7e10 to 7.1e10 but with a 50,000,000 interval and the RAM usage only went up slightly to 1.4GB of RAM max, but often time it hovers around 550MB of RAM.

[Jeff Gilchrist]

Originally Posted by MrRepunit

I am interested. Is it on your usual webpage?

It is but not publicly listed. You can download the core2 optimized version for Windows 64bit here:
http://gilchrist.ca/jeff/factoring/wilsontest_win64.zip

[MrRepunit]

Originally Posted by Jeff Gilchrist

I have 8GB on my system, but I tried running the tool with the full 7e10 to 8e10 start and finish with a 20,000,000 interval and it seemed to be using about 1.2GB of RAM. Now I'm trying just 7e10 to 7.1e10 but with a 50,000,000 interval and the RAM usage only went up slightly to 1.4GB of RAM max, but often time it hovers around 550MB of RAM.

I just took a memory profile of running over 2 hours (the test itself was running much longer before I started the profiling) the range 1e9 to 5e9 using an interval of 50000000 primes. It is normally around 3GB but with lots of peaks up to 4GB. So one can say that using an interval of k*10^7 the program uses at maximum k*0.8 GB RAM.

(If someone is interested in the profiling script I can upload it.)

[fivemack]

The memory usage is very peaky: the below graph is for running 50G .. 60G with interval= 4e7. X axis is in ten-second units since the program starts, Y axis is memory use in kilobytes.

The drops to ~400M use occur at the start of intervals; you can see three phases for each interval - the large-memory phase, an intermediate phase where there's a sort of square-root curve visible in the growth of memory use, and the output phase where the memory use is flat.

The moduli in this graph are [11, 1, 5, 1, 1, 5] (the last 5 is not finished yet); you can clearly see the 1-2-3 timing ratio.