[QUOTE=MyDogBuster;408528]I may as well jump in here.
I'm trying S3 from 1G to 1.1G

Got my newpgen files to n=23

Have latest pfgw 3.7.10
Have srsieve
Have srbsieve dated 8/15/15

SRBsieve started and removed from all 23 files
I then got the message

Started recovery at Phase 0 with 4247788 terms
ERROR m 1e10: argument out of range

My ini file looks like:

base=3
mink=1000000001
maxk=1100000000
c=1
npgfile=1,n1.log
npgfile=2,n2.log
npgfile=3,n3.log
npgfile=4,n4.log
npgfile=5,n5.log
npgfile=6,n6.log
npgfile=7,n7.log
npgfile=8,n8.log
npgfile=9,n9.log
npgfile=10,n10.log
npgfile=11,n11.log
npgfile=12,n12.log
npgfile=13,n13.log
npgfile=14,n14.log
npgfile=15,n15.log
npgfile=16,n16.log
npgfile=17,n17.log
npgfile=18,n18.log
npgfile=19,n19.log
npgfile=20,n20.log
npgfile=21,n21.log
npgfile=22,n22.log
npgfile=23,n23.log
phase=100,33000,70000
phase=335,33000,140000
phase=674,33000,315000
phase=898,33000,552500
phase=1120,33000,425000
phase=1792,33000,977500
phase=2240,33000,595000
phase=2636,33000,1080000
phase=3512,33000,2790000
phase=4364,33000,4410000
phase=5205,33000,4950000
phase=6948,33000,10620000
phase=8624,33000,18000000
phase=10288,33000,31410000
phase=13703,20000,91800000
phase=17027,15000,154800000
phase=20310,15000,190224368
phase=25000,10000,360000000

???[/QUOTE]

I'm using PFGW 3.7.9
srbsieve dated 15 august 2015 (latest version)

and this is how my ini looks:

base=3
mink=1000000001
maxk=1100000000
c=1
npgfile=1,1_.log
npgfile=2,2_.log
npgfile=3,3_.log
npgfile=4,4_.log
npgfile=5,5_.log
npgfile=6,6_.log
npgfile=7,7_.log
npgfile=8,8_.log
npgfile=9,9_.log
npgfile=10,10_.log
npgfile=11,11_.log
npgfile=12,12_.log
npgfile=13,13_.log
npgfile=14,14_.log
npgfile=15,15_.log
npgfile=16,16_.log
npgfile=17,17_.log
npgfile=18,18_.log
npgfile=19,19_.log
npgfile=20,20_.log
npgfile=21,21_.log
npgfile=22,22_.log
npgfile=23,23_.log
phase=100,33000,70000
phase=335,33000,140000
phase=674,33000,315000
phase=898,33000,552500
phase=1120,33000,425000
phase=1792,33000,977500
phase=2240,33000,595000
phase=2636,33000,1080000
phase=3512,33000,2790000
phase=4364,33000,4410000
phase=5205,33000,4950000
phase=6948,33000,10620000
phase=8624,33000,18000000
phase=10288,33000,31410000
phase=13703,20000,91800000
phase=17027,15000,154800000
phase=20310,15000,190224368
phase=25000,10000,360000000

I doesn't get the message that you get MyDogBuster. However you also has more candidates in your recovery count, than should excist after n=23 is removed. Did you stop srbsieve?

After having removed from n=1 to n=23, I have 3998517 candidates remaining and srbsieve starts processing those without problems.

Could you try and run the range with my .ini settings and see if it does make a difference?

Could you try and run it in a new folder where srbsieve and pfgw aswell srsieve has full rights and see if that makes a difference?

[QUOTE]After having removed from n=1 to n=23, I have 3998517 candidates remaining and srbsieve starts processing those without problems.

Could you try and run the range with my .ini settings and see if it does make a difference?
After having removed from n=1 to n=23, I have 3998517 candidates remaining and srbsieve starts processing those without problems.

Could you try and run the range with my .ini settings and see if it does make a difference?

Could you try and run it in a new folder where srbsieve and pfgw aswell srsieve has full rights and see if that makes a difference? [/QUOTE]I too had 3998517 candidates remaining. (read the wrong number)

Ran with your ini and got the same error

All programs have full rights.

I'm not familiar with pfgw. Does it need an ini file?

[QUOTE=MyDogBuster;408539]I'm not familiar with pfgw. Does it need an ini file?[/QUOTE]

This is weird. I'm all out of ideas. PFGW64 (I recon that you use 64 bit) should not need an ini file (created by you) to run, srbsieve should handle all the settings, according to your srbsieve.ini file.

I just completed the same test, using PFGW 3.7.10 and still no errors. What puzzles me is that srbsieve seems to write a checkpoint file before fininshing removing n=21, n=22 and n=23 from the NewPGen files.

Maybe it is a good idea to post your srbsieve.ckpt file, it may mean something to Rogue :smile:

[QUOTE]Maybe it is a good idea to post your srbsieve.ckpt file, it may mean something to Rogue :smile: [/QUOTE]

chpt file looks like this

phase=1
completedStep=3
completedMaxK=1001653986
completedKCount=33000
inProgressKCount=33000

I'm pretty sure [CODE]ERROR m 1e10: argument out of range[/CODE] goes with srsieve. There's this line in the main.cpp code: [CODE]sprintf(command, "srsieve -q -m1e10 -P%"PRIu64" -n%d -N%d -w sieve.in", max_p, n_min, n_max);[/CODE]

If I remember, that command suppresses the printing of any found factors to the screen unless they're greater than the value provided (here, 1e10). If your -P value is less than 1e10, it basically guarantees silent operation. With that said, no idea why it would throw an error.

Found the problem. Had an old version of srsieve. Thanks Kenneth and Ben

[QUOTE=MyDogBuster;408551]Found the problem. Had an old version of srsieve. Thanks Kenneth and Ben[/QUOTE]

Great :tu:

Looking forward to see how fast you can complete the range :smile:

Next problem. Error open file sr_b.pfgw I have a file sr_3.pfgw. ??

Try deleting the checkpoint file and make sure you're starting fresh?

Started fresh. Still asking for sr_b.pfgw.

sr_3.pfgw keeps coming and going and it looks like it's working.

SRSieve is removing candidates and every once in a while I get a phase progress report.

Also seeing "no factoring at all, not even trivial division"

That last line is from pfgw with "-f0" being passed to it. No idea on the rest. Sorry.

It was something like in my post

[URL]http://www.mersenneforum.org/showpost.php?p=408053&postcount=127[/URL]

Solution: dont use any version with checkpoint :)

As for the missing ( and ), I'll add them. Without them you might get a memory exception.

Yes, the checkpoint version doesn't work. I have been way to busy to test it. Please use the last version I posted before adding checkpoint code.

If it stops for some reason while sieving or running pfgw, you can recover manually, but then can't use srbsieve to complete the range.

[QUOTE=rogue;408570]As for the missing ( and ), I'll add them. Without them you might get a memory exception.[/QUOTE]

I'll be curious to see if I was right or not. I was playing with it in python, and with the parentheses where I thought they should go, it threw an index out of bounds error. With them where they are now, it worked flawlessly. But I'm not a solid programmer, so it could easily have been a mistake I made somewhere.

[QUOTE=wombatman;408575]I'll be curious to see if I was right or not. I was playing with it in python, and with the parentheses where I thought they should go, it threw an index out of bounds error. With them where they are now, it worked flawlessly. But I'm not a solid programmer, so it could easily have been a mistake I made somewhere.[/QUOTE]

I try to edit code, add ( ) as you say ,and compile again under Linux
Compilation process is fine but srbieve crash at start...

[QUOTE]~/test# ./srbsieve
Status (00:00:00): Started with 2501 terms
*** Error in `./srbsieve': double free or corruption (out): 0x00000000023883a0 ***
Aborted
[/QUOTE]

Yeah, looks it goes outside of the allocated memory: [CODE]The most common ways to corrupt heap are:

writing past the end of allocation
freeing memory that you didn't allocate
freeing something twice.[/CODE]

Interesting :)

[QUOTE=pepi37;408593]I try to edit code, add ( ) as you say ,and compile again under Linux
Compilation process is fine but srbieve crash at start...[/QUOTE]

Don't know. I just don't have time to work on this right now.

No worries. I was just looking around to try and understand the code, and that part confused me. It frankly looks like it's right as is, and I was wrong about needing to add the parentheses.

Just a heads up for everyone considering to do new bases. I'm currently very far in the process of creating optimized srbase.ini files for ALL partially and ALL untested bases. I expect within the next 6 weeks, to be able to post the srbsieve.ini files needed to run the untested and partially tested bases :smile:

[QUOTE=KEP;412447]Just a heads up for everyone considering to do new bases. I'm currently very far in the process of creating optimized srbase.ini files for ALL partially and ALL untested bases. I expect within the next 6 weeks, to be able to post the srbsieve.ini files needed to run the untested and partially tested bases :smile:[/QUOTE]

This is not gonna happen, within the 6 weeks as hoped. It just simply takes too long to sieve to optimal sievedepth for each phase of S1023. As soon as each phase of S1023 has reached optimal sievedepth, then it is almost as little as a few hours of copy paste, before the .ini files needed to run srbsieve, is all created. [U][B]New ETA is somewhere right after christmas.[/B][/U]

[B]@Rogue:[/B] Could I ask, that the next release (hopefully completed before I releases the 321 srbsieve.ini files I'm currently creating) of srbsieve as Step 1, test for even bases k=1 for all NewPGen sieved n's? ... NewPGen is a great program in many ways, but for some weird reason, all bases fail to contain k=1 in the fixed n sieve and therefor k=1 could have a prime for n<=NewPGen nMax. It's a minor thing, I know, but people who aren't used to work with NewPGen or srbsieve, cannot be expected to be able to figure out, that k=1 may hold 2 primes or still remain at n=nMax (usually n=25K), even though k=1 has a prime wich NewPGen could have found if NewPGen did not avoid k=1 in NewPGens sieve.

[QUOTE=KEP;415273]This is not gonna happen, within the 6 weeks as hoped. It just simply takes too long to sieve to optimal sievedepth for each phase of S1023. As soon as each phase of S1023 has reached optimal sievedepth, then it is almost as little as a few hours of copy paste, before the .ini files needed to run srbsieve, is all created. [U][B]New ETA is somewhere right after christmas.[/B][/U]

[B]@Rogue:[/B] Could I ask, that the next release (hopefully completed before I releases the 321 srbsieve.ini files I'm currently creating) of srbsieve as Step 1, test for even bases k=1 for all NewPGen sieved n's? ... NewPGen is a great program in many ways, but for some weird reason, all bases fail to contain k=1 in the fixed n sieve and therefor k=1 could have a prime for n<=NewPGen nMax. It's a minor thing, I know, but people who aren't used to work with NewPGen or srbsieve, cannot be expected to be able to figure out, that k=1 may hold 2 primes or still remain at n=nMax (usually n=25K), even though k=1 has a prime wich NewPGen could have found if NewPGen did not avoid k=1 in NewPGens sieve.[/QUOTE]

I do not understand your request, please restate.

k=1 does not need to be tested for any bases. The script can start at k=2.

[QUOTE=gd_barnes;415326]k=1 does not need to be tested for any bases. The script can start at k=2.[/QUOTE]

I think I remember that you told me the same thing, quite a few years ago. However, since 1*1018^1+1 is prime, why shouldn't it be part of the conjecture?

@Rogue: I'll get back to you, if it turns out that k=1 has to be tested if k=1 isn't GFN or trivial, otherwise just forget what I previously posted :smile:

[QUOTE=KEP;412447]Just a heads up for everyone considering to do new bases. I'm currently very far in the process of creating optimized srbase.ini files for ALL partially and ALL untested bases. I expect within the next 6 weeks, to be able to post the srbsieve.ini files needed to run the untested and partially tested bases :smile:[/QUOTE]

Last and final delay.

I've made huge progress over the last month. However, IT WILL NOT be untill late january, before I have the 70% sievedepth calculated srbsieve.ini files created.

I currently needs to sieve 3 more phases for S1023 and from there on, I need about 5 reallife days of sieving (~8000 remaining phases for all other remaining untested/partially tested bases) to p=1M, to account for the different removal rate of composites.

As soon as the final sieving of the 3 remaining phases is done and the sieving of the remaining phases (all other untested/partially tested bases) to p=1M is done, then it will be only a matter of days. I really hope that it will not be as late as january 31st 2016, however do not expect any srbsieve.ini files for currently untested/partially tested bases prior to that date.

If you consinder using srbsieve, now is time to get acquainted with how NewPGen and NewPGen's fixed n function works. Also it may be time that Gary and Ian thinks about how to get the remaining untested bases for conjectured k<=k-limit-of-sr2sieve started. All conjectures less than the limits of sr2sieve, may benefit from running srbsieve to a low n and therefrom switch to sieving using sr2sieve instead of srsieve.

Take care and happy new year.

[QUOTE=KEP;415422]I think I remember that you told me the same thing, quite a few years ago. However, since 1*1018^1+1 is prime, why shouldn't it be part of the conjecture?

@Rogue: I'll get back to you, if it turns out that k=1 has to be tested if k=1 isn't GFN or trivial, otherwise just forget what I previously posted :smile:[/QUOTE]

k=1 is ALWAYS GFN or trivial. That's why we don't consider it. Yes some bases will have a GFN prime like S1018 but many will not. See the pages where it says "k=1 is a GFN with no known prime.". S1014 and S1016 are examples. See a list of known GFN primes for bases <=1030 up to n=2^17 at [URL]http://www.noprimeleftbehind.net/crus/GFN-primes.htm[/URL]. See that S1014 and S1016 have none. The largest known prime for bases <= 1030 is 150^2048+1. There are unlikely to be any others for n>2^17. (I believe that other projects have searched these to n=2^20.)

[QUOTE]
Just a heads up for everyone considering to do new bases. I'm currently very far in the process of creating optimized srbase.ini files for ALL partially and ALL untested bases. I expect within the next 6 weeks, to be able to post the srbsieve.ini files needed to run the untested and partially tested bases
[/QUOTE][QUOTE=KEP;420532]Last and final delay.

I've made huge progress over the last month. However, IT WILL NOT be untill late january, before I have the 70% sievedepth calculated srbsieve.ini files created.

I currently needs to sieve 3 more phases for S1023 and from there on, I need about 5 reallife days of sieving (~8000 remaining phases for all other remaining untested/partially tested bases) to p=1M, to account for the different removal rate of composites.

As soon as the final sieving of the 3 remaining phases is done and the sieving of the remaining phases (all other untested/partially tested bases) to p=1M is done, then it will be only a matter of days. I really hope that it will not be as late as january 31st 2016, however do not expect any srbsieve.ini files for currently untested/partially tested bases prior to that date.

If you consinder using srbsieve, now is time to get acquainted with how NewPGen and NewPGen's fixed n function works. Also it may be time that Gary and Ian thinks about how to get the remaining untested bases for conjectured k<=k-limit-of-sr2sieve started. All conjectures less than the limits of sr2sieve, may benefit from running srbsieve to a low n and therefrom switch to sieving using sr2sieve instead of srsieve.

Take care and happy new year.[/QUOTE]


I don't understand your first post here and then your update. So are you really testing ALL of the remaining bases, for example R280 and S280? I hope not. The prime files alone would be petabytes. I don't plan to administer such monstrosities.

If you are only doing S1023, then that is OK.

[QUOTE=gd_barnes;420575]I don't your first post here and then your update. So are you really testing ALL of the remaining bases, for example R280 and S280? I hope not. The prime files alone would be petabytes. I don't plan to administer such monstrosities.

If you are only doing S1023, then that is OK.[/QUOTE]

I'm only doing S1023, but from that base I calculate the sievedepth for 319 other partially or untested bases and create the srbsieve.ini files for each of those bases. So even though I'm only testing S1023, once I have the 70% sievedepth for each of the 36 phases, I can calculate the 70% sievedepth for each 21 to36 phases for the remaining unstarted and partially tested bases. This in other words means, that once I reach the 70% sievedepth for the 3 currently unsieved phases for S1023, that I will be able to create the nescessary srbsieve.ini files for each remaining untested or partially tested base. Once I have the ini files, I will make them publicly available and then who knows what will happen. But as you said, S280 and R280 will not be within our reaching point anytime in the near future, however with the flood of ini files for use with srbsieve, it is not unreasonable to consider most conjectures for conjectured k<=10M possible to get to n=25K, hence it may be a good idea to have some sort of battleplan on how high to test using srbsieve and what to test like Ian has done in the recommended thread.

Thanks for the clarification. Whew. :smile:

[QUOTE=gd_barnes;420574]k=1 is ALWAYS GFN or trivial. That's why we don't consider it. Yes some bases will have a GFN prime like S1018 but many will not. See the pages where it says "k=1 is a GFN with no known prime.". S1014 and S1016 are examples. See a list of known GFN primes for bases <=1030 up to n=2^17 at [URL]http://www.noprimeleftbehind.net/crus/GFN-primes.htm[/URL]. See that S1014 and S1016 have none. The largest known prime for bases <= 1030 is 150^2048+1. There are unlikely to be any others for n>2^17. (I believe that other projects have searched these to n=2^20.)[/QUOTE]

Thanks for the reexplanation. All srbsieve.ini files (total of 318) have now been corrected and will (when ready for release) have a minK=2 and a maxK=conjecture-k minus one (or maxK=1G, if conjectured k>1G).

Happy new year each and everyone :smile:

I have rewritten the recovery logic of srbsieve. There are a few things of note:
[LIST][*]it can only recover if you stop while srsieve or pfgw is running[*]it won't update pl_remain.txt or pl_prime.txt until the entire phase is complete[/LIST]
I have done zero testing of the changes, but the logic is much simpler than what it was. If you happen to stop the program at any other point other than when one of those programs is running, srbsieve will require you to restart the step, which could cost you a lot of time. Note that 99.9999% of the time is spent in srsieve and pfgw, so I woudn't worry too much about that being a problem.

In theory one could use this to extend a search. For example, if you complete a large range to n=1000 then want to pass ranges on to others to get to n=25000 that should be possible.

I was able to do some testing today. I've worked out most of the bugs, but I don't know if all of the bugs are addressed. There are a few areas that I need to test yet. So far, I have been able to cleanly recover when in the first phase. I need to test recovery in subsequent phases and recovery when a phase is broken up into multiple sub-phases.

[QUOTE=rogue;424859]I was able to do some testing today. I've worked out most of the bugs, but I don't know if all of the bugs are addressed. There are a few areas that I need to test yet. So far, I have been able to cleanly recover when in the first phase. I need to test recovery in subsequent phases and recovery when a phase is broken up into multiple sub-phases.[/QUOTE]

I will try to test latest srbsieve with low K tested conjuncture...
Few tests I test before give me srbsieve crash at second stage...

Due to a request, I modified the program to support base 2.

[QUOTE=rogue;434369]Due to a request, I modified the program to support base 2.[/QUOTE]
btw, are there any docs with explanations, for phases etc. and other thingies that can be added to the ini file?

[QUOTE=LaurV;434372]btw, are there any docs with explanations, for phases etc. and other thingies that can be added to the ini file?[/QUOTE]

There are other posts in this thread that explain the ini file and some of them have examples.

pepi37 reported a problem. Hopefully the attached fixes it.

[QUOTE=rogue;434400]There are other posts in this thread that explain the ini file and some of them have examples.

pepi37 reported a problem. Hopefully the attached fixes it.[/QUOTE]

not fixing anything: it looks like you gave link to old file
same error :(

[QUOTE=pepi37;434403]not fixing anything: it looks like you gave link to old file
same error :([/QUOTE]

Very perceptive of you. It was a test. :cmd:

Not a test a test of your ability to perceive my mistakes though. :ouch2: It was a test of my ability to zip the correct file.

[QUOTE=rogue;434415]Very perceptive of you. It was a test. :cmd:

Not a test a test of your ability to perceive my mistakes though. :ouch2: It was a test of my ability to zip the correct file.[/QUOTE]

It look like you will make it from third shoot :)
Or you are very sleepy , your eyes is closed.. .you sleep


Once more, same error :)
You zipped test of the test, I looks like :)

I found the problem. I was comparing to the numeric value of 1, not the character value of '1'.

Ok, let see how it works now :-)

And it is work perfectly!
Thanks Rogue

Question: isn't the file called "pl_remain.tmp" supposed to get smaller after every phase finishes? (as the new primes are reduced from it?)

I scheduled some work, just for a test, but I miscalculated the time it will take, by a factor of 5 or so, and now it seems that it will take another 2-3 days, from a total of about 5. The work is split in about 9 phases, but it seems that after every phase, I get a bigger "remaining" file, it looks like only the primes found in the [U]last[/U] phase are eliminated, and not all of them.

The point being, I am not using the last srbsieve, but the one I got when I started playing with it, and I see here some upgrades in the meantime, should I stop, erase everything, [U]download the last srbsieve[/U] (and the last pfgw64, I still use 3.7.7, and here I am getting the "log file not found" warning at every bunch) and start from scratch? Or let it continue? (i.e. at the end it will still do some magic with the pl_remaining file?)

[QUOTE=LaurV;434531]The point being, I am not using the last srbsieve, but the one I got when I started playing with it, and I see here some upgrades in the meantime, should I stop, erase everything, [U]download the last srbsieve[/U] (and the last pfgw64, I still use 3.7.7, and here I am getting the "log file not found" warning at every bunch) and start from scratch? Or let it continue? (i.e. at the end it will still do some magic with the pl_remaining file?)[/QUOTE]

Download the latest version of PFGW, srbsieve and srsieve. It sure seems like something is not working together the way things should.

If your pl_prime.txt file grows in size after completion of each phase, then your pl_remain.txt should decrease in size.

I'm pretty sure if you download all the latest versions of each of the previously mentioned programs, that your problems will go away. :smile:

EDIT: No there is nothing magic done to pl_remain.txt after completion of the final phase. The magic happens after completion of each phase and not only after completion of the final phase.

[QUOTE=LaurV;434531...

The point being, I am not using the last srbsieve, but the one I got when I started playing with it, and I see here some upgrades in the meantime, should I stop, erase everything, [U]download the last srbsieve[/U] (and the last pfgw64, I still use 3.7.7, and here I am getting the "log file not found" warning at every bunch) and start from scratch? Or let it continue? (i.e. at the end it will still do some magic with the pl_remaining file?)[/QUOTE]

As KEP says, download all new: ( why you still using PFGW3.7.7??? )
Download last srbsieve ( from post above yours) since from first one there was about 10 version with little and small bug fixed.

[QUOTE=LaurV;434531]Question: isn't the file called "pl_remain.tmp" supposed to get smaller after every phase finishes? (as the new primes are reduced from it?)

I scheduled some work, just for a test, but I miscalculated the time it will take, by a factor of 5 or so, and now it seems that it will take another 2-3 days, from a total of about 5. The work is split in about 9 phases, but it seems that after every phase, I get a bigger "remaining" file, it looks like only the primes found in the [U]last[/U] phase are eliminated, and not all of them.

The point being, I am not using the last srbsieve, but the one I got when I started playing with it, and I see here some upgrades in the meantime, should I stop, erase everything, [U]download the last srbsieve[/U] (and the last pfgw64, I still use 3.7.7, and here I am getting the "log file not found" warning at every bunch) and start from scratch? Or let it continue? (i.e. at the end it will still do some magic with the pl_remaining file?)[/QUOTE]

Use the latest srbsieve. If pl_remain.tmp never decreases in size, then there is a problem, but it could be due to you using a buggy version of srbsieve.

[QUOTE=rogue;434556]Use the latest srbsieve. If pl_remain.tmp never decreases in size, then there is a problem, but it could be due to you using a buggy version of srbsieve.[/QUOTE]
Switching to last version of srbsieve solved the problem, also the "missing pfgw log file" warning disappeared. I am now playing with a smaller assignment to see how it is working. It seems ok.

The older srbsieve indeed was not decreasing the size of "pl_remain.txt", it was only eliminating the k's in the current phase, starting from the full file every time, therefore the size of the file was increasing.

Thanks to pepi, more bugs are squashed. Recovery seems to be working well right now.

Still not working as well... you got PM

I don't know exactly what happened, but it must have been stopped at some really odd time during the run since pl_remain.txt doesn't exist. I made a change that will force starting from scratch if that file is not found.

I added a print out of the k remaining, but it is only printed after a phase is completed.

[QUOTE]Phase 5: Processing k from 302 to 964
command: srsieve -q -m1e10 -P58181000 -n1281 -N2560 -w sieve.in
command: pfgw64 -Cquiet -f0 sr_b.pfgw > nul
PFGW Version 3.7.10.64BIT.20150809.Win_Dev [GWNUM 28.6]

No factoring at all, not even trivial division
ABC File Processing for at most 1 Primes


1 file(s) moved.
command: pfgw64 -Cquiet -tm sr_b.prp > nul
PFGW Version 3.7.10.64BIT.20150809.Win_Dev [GWNUM 28.6]


1 file(s) moved.
Phase 6: Processing k from 302 to 964
command: srsieve -q -m1e10 -P73588000 -n2561 -N3072 -w sieve.in
command: pfgw64 -Cquiet -f0 sr_b.pfgw > nul
PFGW Version 3.7.10.64BIT.20150809.Win_Dev [GWNUM 28.6]

No factoring at all, not even trivial division
ABC File Processing for at most 1 Primes[/QUOTE]

I dont see the number of remain candidate, do you see it?

KEP reported a bug with Sierpinski mew base searching. This bug causes srbsieve to miss MOBs. Fortunately you don't need to re-run an entire range (or generate inputs from newpgen) to find the missing values. Run the new version and it will output an updated pl_MOB.txt.. If any of those k show up in your pl_remain.txt file, remove them.

I made a small change to address a problem found by MisterBitcoin. newpgen will sieve to sqrt(largest candidate) and srbsieve checks for that, but it turns out that newpgen will sometimes sieve to the largest prime less than sqrt(largest candidate) and not the smallest prime greater than sqrt(largest candidate). With the current behavior, srbsieve terminates with an error. I added a check to allow for the difference so that srbsieve will not terminate. The check isn't perfect, but for the large ranges of k being tested, it shouldn't be a problem. There is a very, very slim chance that someone could terminate newpgen early and srbsieve will allow for the file without giving the error. By early, I mean stopping newpgen within a few hundred primes of which it would sieve to, which could probably only happen if someone were deliberately trying to break it.

I found an other bug, this time its something small.

These progress on phase 8 was not 0,23 pct. This is happening because the progress for phase 8 is using the same sub-phase-amount from phase 2. :)

[QUOTE=MisterBitcoin;454627]I found an other bug, this time its something small.

These progress on phase 8 was not 0,23 pct. This is happening because the progress for phase 8 is using the same sub-phase-amount from phase 2. :)[/QUOTE]

I don't see a problem in the code. Did you stop and restart the range at some point? Can you e-mail me the input files you used?

[QUOTE=rogue;457698]I don't see a problem in the code. Did you stop and restart the range at some point? Can you e-mail me the input files you used?[/QUOTE]

All ready too late. But I can see the same think on S7/15. So far I got it right:

The progress that srbsieve prints out it using the amount of sub-phases. You will have a few hundred sub-phases in phase 2, but only the few in phase 16.

The process from S7 and S15 is running with any disruption.

Anyway: k´s remain print out function isn´t working. Pepi just pointed it out a few month ago.
These thinks are optional, personally I don´t need them. Just nice to have. :smile:

[QUOTE=MisterBitcoin;457701]All ready too late. But I can see the same think on S7/15. So far I got it right:

The progress that srbsieve prints out it using the amount of sub-phases. You will have a few hundred sub-phases in phase 2, but only the few in phase 16.

The process from S7 and S15 is running with any disruption.

Anyway: k´s remain print out function isn´t working. Pepi just pointed it out a few month ago.
These thinks are optional, personally I don´t need them. Just nice to have. :smile:[/QUOTE]

I see why it isn't outputting the remaining k's. It builds a string with the information but doesn't dump it to the screen.

I still need to see some files for the other issue. I'm not seeing any problems in the code yet.

I just discovered that Ken Brazier has a program called nsieve64. This can work like newpgen, but is 64-bit. As it is source, it can be compiled on other platforms. It claims to be faster than newpgen.

I'll take a look and see if it can be multi-threaded. It could also benefit by accepting parameters the way other sieving programs do.

It does seem to have other issues as it doesn't correctly parse inputs > 32 bits, but that should be easy to address.

[QUOTE=rogue;466272]I just discovered that Ken Brazier has a program called nsieve64. This can work like newpgen, but is 64-bit. As it is source, it can be compiled on other platforms. It claims to be faster than newpgen.

I'll take a look and see if it can be multi-threaded. It could also benefit by accepting parameters the way other sieving programs do.

It does seem to have other issues as it doesn't correctly parse inputs > 32 bits, but that should be easy to address.[/QUOTE]

Can you build exe file: to start learning what this program can make?
I try to build it, but got many errors :(

[QUOTE=pepi37;466284]Can you build exe file: to start learning what this program can make?
I try to build it, but got many errors :([/QUOTE]

If you have Windows 10, it will build in the Ubuntu shell with
[CODE]g++ nsieve64.cpp -o nsieve64[/CODE]

And testing with
[CODE]./nsieve64 -s 6209,800000000,800000001 -N 6299 -o nsieve64_testoutput.txt -p 2,1000000000000[/CODE]

seems to be working. One note: you cannot currently use abbreviated inputs like 1e12; they must be typed out fully.

[QUOTE=wombatman;466286]If you have Windows 10, it will build in the Ubuntu shell with
[CODE]g++ nsieve64.cpp -o nsieve64[/CODE]

And testing with
[CODE]./nsieve64 -s 6209,800000000,800000001 -N 6299 -o nsieve64_testoutput.txt -p 2,1000000000000[/CODE]

seems to be working. One note: you cannot currently use abbreviated inputs like 1e12; they must be typed out fully.[/QUOTE]

I recommend building with this:

g++ -O3 -m64 nsieve64.cpp -lstdc++ -o nsieve

It is currently slower than fermfact, but I found one optimization that seems to make it faster for multiple n, but I need to run some side-by-side comparisons with fermfact. If someone can build and compare with newpgen (for a single n), that would be great. Note that newpgen stops automatically when p > sqrt(maxk*2^n+1). nsieve does not stop automatically.

I also intend to modify nsieve to be multi-threaded, but I'm concerned about the performance it could take.

I just ran one test. nsieve and newpgen do not produce the same outputs. I assume that nsieve must have some bugs in it. That will take time to resolve.

Yeah, I was just about to post the same thing. Using n=30 with k=8e8-800000500, I get 12 candidates remaining from newpgen. Nsieve's last output on screen shows 8 remaining, and the output file has the proper header and then nothing else.

I got empty output from your build Rogue

[QUOTE=pepi37;466290]I got empty output from your build Rogue[/QUOTE]

It isn't my code. I'm trying to find contact information for Ken Brazier as he wrote it although I might be able to debug without his help.

It appears to be a miscompile on Windows with mingw. The outputs on my Mac are different and appear to be correct, although I haven't run any comparisons.

Are you able to compile it on a Linux system at all and confirm whether it mismatches?

[QUOTE=rogue;466377]It appears to be a miscompile on Windows with mingw. The outputs on my Mac are different and appear to be correct, although I haven't run any comparisons.[/QUOTE]

The first test I ran shows non-matching output on my Mac. I don't think this is a miscompile. It is either bad assembly code or the prime sieve is missing primes.

nsieve is only for base 2, so it isn't too helpful for this project.

I ran into an issue with srbsieve. When it breaks up large ranges for PRP testing it is skipping some of those ranges. I don't how that is happening, but I'm looking into it.

This means that if you did a large range of k with srbsieve it is likely that some of the k in pl_remain.txt have a prime for n < 25000. This has a definite impact on R3 and S3, but I can't speak for other bases. If you have used srbsieve then you should run from n=1 to 25000 for the k in pl_remain.txt to find any primes that it missed. There are no issues with the numbers in pl_prime.txt. They will be prime. Unfortunately this is going to create a lot of work for me due to the work I have done on S3.

Hopefully I will have time between Christmas and New Year's to track down the problem and fix it.

Does this problem concern all versions of srbsieve or only those that did checkpoint and allowed to resume? I used for my R3 ranges, the version that did not support stopping and starting again - in other words no resume was allowed.

[QUOTE=KEP;474095]Does this problem concern all versions of srbsieve or only those that did checkpoint and allowed to resume? I used for my R3 ranges, the version that did not support stopping and starting again - in other words no resume was allowed.[/QUOTE]

Possibly. IMO, if you run pl_remain up to 10000 and find no new primes, then the likelihood of an issue is extremely small. I've run some tests and see no patterns that tell me where the problem is.

[QUOTE=rogue;474097]Possibly. IMO, if you run pl_remain up to 10000 and find no new primes, then the likelihood of an issue is extremely small. I've run some tests and see no patterns that tell me where the problem is.[/QUOTE]

I´ll rerun my pl_remain files for S7 and S15 shortly. I´ll report here if I found any new primes in that range, I´ll use cllr.
Same will go on with S3, on this base I´ll wait until you have more infos on it.

I have checked S15 the range from n=1-250 with LLR, no prime found.
I´ll now take a look onto S3; taking the range 17G-19G which the same n-range.
I´ll expend the n-range when you have more information's.

Although I haven't proven it to myself yet, I'm fairly certain certain that the problem is with resuming from a checkpoint, specifically when the phase has to be broken into sub-phases. For example, if there are more remaining k for a phase than the number configured per sub-phase. For example, one of the range here would be from n=1792 thru n=2240:

phase=1792,33000,977500
phase=2240,32000,595000

If by the time you reach n=1792 that there are more than 33000 k remaining, then it will break up the ranges into sub-phases of 33000 k each. I suspect that when srbsieve is stopped and restarted in the middle of a sub-phrase that it loses track of which k it has processed and ends up skipping k. Once the number of k remaining falls below the number of k per sub-phase, the problem won't be manifested.

If I am correct, it means that it only impacts the tested range if srbsieve was stopped and restarted before there were too few k remaining to trigger multiple sub-phases. For R3 and S3, that would be about n=8624 which would be phase 13.

This shouldn't be too difficult to prove and if I'm right, the retesting of remaining k for small n shouldn't take too long, about 3 days per 1G range.

[QUOTE=rogue;474174]Although I haven't proven it to myself yet, I'm fairly certain certain that the problem is with resuming from a checkpoint, specifically when the phase has to be broken into sub-phases. For example, if there are more remaining k for a phase than the number configured per sub-phase. For example, one of the range here would be from n=1792 thru n=2240:

phase=1792,33000,977500
phase=2240,32000,595000

If by the time you reach n=1792 that there are more than 33000 k remaining, then it will break up the ranges into sub-phases of 33000 k each. I suspect that when srbsieve is stopped and restarted in the middle of a sub-phrase that it loses track of which k it has processed and ends up skipping k. Once the number of k remaining falls below the number of k per sub-phase, the problem won't be manifested.

If I am correct, it means that it only impacts the tested range if srbsieve was stopped and restarted before there were too few k remaining to trigger multiple sub-phases. For R3 and S3, that would be about n=8624 which would be phase 13.

This shouldn't be too difficult to prove and if I'm right, the retesting of remaining k for small n shouldn't take too long, about 3 days per 1G range.[/QUOTE]

If thats he cause I dont need to test my results, because I never paused or stopped while it was running.
The S3 range just passed n=100 and, like expected, no primes found in that range.

[QUOTE=MisterBitcoin;474176]If thats he cause I dont need to test my results, because I never paused or stopped while it was running.
The S3 range just passed n=100 and, like expected, no primes found in that range.[/QUOTE]

If each 1G range has about 6500 remaining k after being tested to n=25000, you should be good.

[QUOTE=rogue;474193]If each 1G range has about 6500 remaining k after being tested to n=25000, you should be good.[/QUOTE]

When you guys send me the remaining k's files I visually check them to make sure that the number of k's remaining looks normal. Nothing has looked unusual so far with these huge efforts that you guys have done with the exception of the k=119G-120G range that you sent me.

The number of k's remaining increases as the k's get larger at a surprisingly fast rate due to the smallest tests at n=1, 2, 3, etc. being significantly larger hence fewer such primes. For example for Bitcoin's range around k=17G-27G, there was an average of ~6100-6200 k's remaining for each k=1G range at n=25K. As you found for k>100G, there are about ~6500 k's remaining for the same range.

For k=~5G-10G, 5600-5800 k's remaining was typical for each 1G range.

The major change in this new release is that it supports ABCD files as the input file format in addition to newpgen format. The ABCD format is about one-fourth the size of the newpgen format which is great if you have limited disk space. With S3, the newpgen files for a 4G range take up over 80 GB of disk space, which is a crunch on a 250 GB SDD.

Another change is that it will not generate the pl_remain.txt file until all input files are processed, which saves a lot time for the first phase.

If you really want to be anal, you can us "abcdfile=" instead of "npgfile=" in srbsieve.ini. They are interchangable so you can use abcdfile= with a newpgen formatted file and npgfile= with an ABCD formatted file.

I have not looked into the issue with restarting from within a subphase, but it is on my radar.

The previously attached file was a debug build which requires other libraries. This is a release build so if you have already been running the older release, it won't require you to install more stuff.

I know now one reason for the missed primes. I have seen an issue with my laptop that leads to "low on memory" warnings. This is happening in pfgw and causes pfgw to terminate before it processes all candidates in the ABC file. srbsieve doesn't know that pfgw had an error, thus doesn't know that it didn't complete and therefore continues on its merry way to the next phase/subphase.

What is really annoying is that I can't determine if this is really a bug in pfgw or if it is elsewhere (Windows, laptop memory, etc) If this were in pfgw, then I would expect a lot of people to complain about it, but nobody has. When I get these low memory warnings they come in bunches, but go away when I restart the laptop. Note that this laptop has 32 GB and it rarely goes uses more than 16 GB of memory. If pfgw had a memory leak then I would expect to see it using lots of memory in Task Manager, but I have never witnessed it using more than a few MB.

If anyone else has seen "low memory warnings" in Windows when running pfgw, I would like to know.