Sieve Benchmark Thread

[Flatlander]

Okay, thanks. I'll try it with the 0.75T range I have left to do and see how much memory it uses.

[axn]

Originally Posted by henryzz

It is possible to change the level of the recombine. Just set a limit on the sieve to shortly after you estimate 1/250(I would recommend 1/300 or less in case of mistakes) are remaining. That could in theory mean its possible to combine really early like 1e6 or something like that.
I will do a test now to see vaguely when.
edit: ~p=4e4 would do the trick nicely

How much time will this save? By my reckoning, you won't actually save any time at all.

[Flatlander]

Originally Posted by axn

How much time will this save? By my reckoning, you won't actually save any time at all.

Possibly. Sieving a 0.05T range to 4e4 just took 27 mins, so < 10 hours for 1T. (Compare with c.24hrs to 1G.)
Sieving from 4e4 to 100G is progressing painfully slowly but of course the first bit is always much slower.

I think I'll just stop now and run a whole 1T range, now that I know it will easily fit in 485Mb once sieved to 4e4. I'll post the timings here later.

The NPG help file states:
"NewPGen is happy with lots of k's to sieve - there is nothing to be gained by dividing a range of k's up and sieving each subrange in turn..."
So I'm hoping there will be an increase in efficiency.

[axn]

Originally Posted by Flatlander

Sieving from 4e4 to 100G is progressing painfully slowly but of course the first bit is always much slower.

That's why I am asserting that there'll be no speed gain. In fact, due to the greater IO involved, it might actually be slower.

Originally Posted by Flatlander

The NPG help file states:
"NewPGen is happy with lots of k's to sieve - there is nothing to be gained by dividing a range of k's up and sieving each subrange in turn..."
So I'm hoping there will be an increase in efficiency.

Only true when p >= k range (or maybe range/2). Otherwise there is no increase in efficiency, and might even be slower due to memory pressure (Fast Array vs Normal Array mode).

[Flatlander]

Fair enough.

[The Carnivore]

Originally Posted by Historian

Processor: Pentium 4 3.4 GHz

tpsieve for the variable n-range: 5M p/sec

tpsieve for the variable n-range: 14.1M p/sec using one core of a 2.4 GHz core2duo processor.

[amphoria]

Originally Posted by axn

Only true when p >= k range (or maybe range/2). Otherwise there is no increase in efficiency, and might even be slower due to memory pressure (Fast Array vs Normal Array mode).

This is in fact the problem with only sieving to p=4e4 in the first stage. It leaves about 32.5 M k's and therefore NewPGen uses normal array mode. I hadn't worked out why it was so slow until I stopped the sieve at p=1e6. When restarted NewPGen switched to fast array mode and the removal rate jumped from 40 k/sec to 280 k/sec, If there is an advantage (yet to be proven), then the first stage needs to sieve to much closer to p=1e6.

[Flatlander]

Re. Megabit Twin Sieve
I tried sieving to 10M then again to 100G. The total time was 10-20% slower than letting NPG do it automatically.
I would seem that indeed the 'sweet spot' is above 1G.

Some scribbled timings:
Start to 10M took 15hrs 3m (Leaving 6.1M ks.)
Started NPG again. (Used fast array, 384Mb ram.)
10M-20M took 1hr 24m
20m-100m took 4hrs 25m
100m-500m took 2hrs 41m
500m-100G to 8hrs 33m

Total time c. 32hrs compared with c. 26-28 hrs (iirc) letting NPG do it automatically.

C2Quad Q6700 at stock 2.66GHz. 2Gb ram, NPG memory at maximum 485Mb.

[axn]

Originally Posted by amphoria

This is in fact the problem with only sieving to p=4e4 in the first stage. It leaves about 32.5 M k's and therefore NewPGen uses normal array mode. I hadn't worked out why it was so slow until I stopped the sieve at p=1e6. When restarted NewPGen switched to fast array mode and the removal rate jumped from 40 k/sec to 280 k/sec, If there is an advantage (yet to be proven), then the first stage needs to sieve to much closer to p=1e6.

Hmmm... How much time did the sieve take to get to 4e4? 1e6? 10e6?

I am asking because I am "fairly confident" that I can write a custom sieve that can sieve a range of k's to 1e6 (or even 10e6) much faster than NewPGen can. NewPGen isn't really optimised for the initial sieving.

[axn]

Originally Posted by Flatlander

Some scribbled timings:
Start to 10M took 15hrs 3m (Leaving 6.1M ks.)

Hmmm. I should be able to do better than this. Gotta code it up, though, to know for sure.

[Flatlander]

Originally Posted by axn

Hmmm... How much time did the sieve take to get to 4e4? 1e6? 10e6?...

Originally Posted by Flatlander

... Sieving a 0.05T range to 4e4 just took 27 mins, so < 10 hours for 1T...

All I have is '7hrs 46m to 4e4' jotted on a piece of paper but that sounds a bit quick.

All figures subject to slight variation as NPG had high priority while 4 x LLRNet with low priority were running in the background. (So I didn't waste any cycles when NPG finished overnight.)