Okay, thanks. I'll try it with the 0.75T range I have left to do and see how much memory it uses.

[QUOTE=henryzz;217716]It is possible to change the level of the recombine. Just set a limit on the sieve to shortly after you estimate 1/250(I would recommend 1/300 or less in case of mistakes) are remaining. That could in theory mean its possible to combine really early like 1e6 or something like that.
I will do a test now to see vaguely when.
edit: ~p=4e4 would do the trick nicely[/QUOTE]

How much time will this save? By my reckoning, you won't actually save any time at all.

[QUOTE=axn;217731]How much time will this save? By my reckoning, you won't actually save any time at all.[/QUOTE]
Possibly. Sieving a 0.05T range to 4e4 just took 27 mins, so < 10 hours for 1T. (Compare with c.24hrs to 1G.)
Sieving from 4e4 to 100G is progressing painfully slowly but of course the first bit is always much slower.

I think I'll just stop now and run a whole 1T range, now that I know it will easily fit in 485Mb once sieved to 4e4. I'll post the timings here later.

The NPG help file states:
"NewPGen is happy with lots of k's to sieve - there is nothing to be gained by dividing a range of k's up and sieving each subrange in turn..."
So I'm hoping there will be an increase in efficiency.

[QUOTE=Flatlander;217741]Sieving from 4e4 to 100G is progressing painfully slowly but of course the first bit is always much slower.[/quote]
That's why I am asserting that there'll be no speed gain. In fact, due to the greater IO involved, it might actually be slower.
[QUOTE=Flatlander;217741]
The NPG help file states:
"NewPGen is happy with lots of k's to sieve - there is nothing to be gained by dividing a range of k's up and sieving each subrange in turn..."
So I'm hoping there will be an increase in efficiency.[/QUOTE]

Only true when p >= k range (or maybe range/2). Otherwise there is no increase in efficiency, and might even be slower due to memory pressure (Fast Array vs Normal Array mode).

Fair enough. :smile:

[QUOTE=Historian;217612]
Processor: Pentium 4 3.4 GHz

tpsieve for the variable n-range: 5M p/sec
[/QUOTE]
tpsieve for the variable n-range: 14.1M p/sec using one core of a 2.4 GHz core2duo processor.

[QUOTE=axn;217744]
Only true when p >= k range (or maybe range/2). Otherwise there is no increase in efficiency, and might even be slower due to memory pressure (Fast Array vs Normal Array mode).[/QUOTE]

This is in fact the problem with only sieving to p=4e4 in the first stage. It leaves about 32.5 M k's and therefore NewPGen uses normal array mode. I hadn't worked out why it was so slow until I stopped the sieve at p=1e6. When restarted NewPGen switched to fast array mode and the removal rate jumped from 40 k/sec to 280 k/sec, If there is an advantage (yet to be proven), then the first stage needs to sieve to much closer to p=1e6.

Re. Megabit Twin Sieve
I tried sieving to 10M then again to 100G. The total time was 10-20% slower than letting NPG do it automatically.
I would seem that indeed the 'sweet spot' is above 1G.

Some scribbled timings:
Start to 10M took 15hrs 3m (Leaving 6.1M ks.)
Started NPG again. (Used fast array, 384Mb ram.)
10M-20M took 1hr 24m
20m-100m took 4hrs 25m
100m-500m took 2hrs 41m
500m-100G to 8hrs 33m

Total time c. 32hrs compared with c. 26-28 hrs (iirc) letting NPG do it automatically.

C2Quad Q6700 at stock 2.66GHz. 2Gb ram, NPG memory at maximum 485Mb.

[QUOTE=amphoria;218272]This is in fact the problem with only sieving to p=4e4 in the first stage. It leaves about 32.5 M k's and therefore NewPGen uses normal array mode. I hadn't worked out why it was so slow until I stopped the sieve at p=1e6. When restarted NewPGen switched to fast array mode and the removal rate jumped from 40 k/sec to 280 k/sec, If there is an advantage (yet to be proven), then the first stage needs to sieve to much closer to p=1e6.[/QUOTE]

Hmmm... How much time did the sieve take to get to 4e4? 1e6? 10e6?

I am asking because I am "fairly confident" that I can write a custom sieve that can sieve a range of k's to 1e6 (or even 10e6) much faster than NewPGen can. NewPGen isn't really optimised for the initial sieving.

[QUOTE=Flatlander;218274]Some scribbled timings:
Start to 10M took 15hrs 3m (Leaving 6.1M ks.)
[/QUOTE]

Hmmm. I should be able to do better than this. Gotta code it up, though, to know for sure.

[QUOTE=axn;218276]Hmmm... How much time did the sieve take to get to 4e4? 1e6? 10e6?...[/QUOTE]
[QUOTE=Flatlander;217741]... Sieving a 0.05T range to 4e4 just took 27 mins, so < 10 hours for 1T...[/QUOTE]
All I have is '7hrs 46m to 4e4' jotted on a piece of paper but that sounds a bit quick.

All figures subject to slight variation as NPG had high priority while 4 x LLRNet with low priority were running in the background. (So I didn't waste any cycles when NPG finished overnight.)