Algebraic factors in sieve files

[henryzz]

Can I suggest that a good safety net would be for srsieve to verify its own algebraic factors?

[pepi37]

My friend, with my support write small program 100% based on Batalov script. I have no plan to add more function then only two have implemented: so it will check only npg files, can list algeberic factors, and can save it in factors.txt , prepared for removing with srsieve. Batalov script is still superior because it is way faster, and can check many npg files at once. First , it need more testing, but it exist.

[gd_barnes]

Originally Posted by pepi37

My friend, with my support write small program 100% based on Batalov script. I have no plan to add more function then only two have implemented: so it will check only npg files, can list algeberic factors, and can save it in factors.txt , prepared for removing with srsieve. Batalov script is still superior because it is way faster, and can check many npg files at once. First , it need more testing, but it exist.

I received your program and did some small amount of testing on it with no problems found. I'll do more extensive testing over the next few days. The GUI is nice.

My preference as it has been all along is that srsieve be left alone and for stand-alone scripts such as this to be used to remove algebraic factors after sieving to a nominal limit.

My reasoning is that attempting to integrate algebraic factors code in with all of the complex possible conditions that srsieve can handle is very risky as my testing has clearly demonstrated. It is that integration that adds a large portion of the risk. When a program outside of srsieve can be used to simply remove the factors there is little integrating involved. Then it's just a matter of understanding all of the possible combinations of algebraic factors.

I'm cringing after doing this round of testing on srsieve and there will be far more testing needed after corrections are made.

[gd_barnes]

Originally Posted by henryzz

Can I suggest that a good safety net would be for srsieve to verify its own algebraic factors?

No for several reasons.

1. A program should not have code that has to be checked later on in the program. That is poor coding practice. It just adds an additional layer of code and checking in the program, which if properly coded and tested should not be needed. A proper test plan will verify that all of the factors that it is finding are correct.

2. All testing done so far has indicated that all of the algebraic factors that it is finding are correct factors. Mark also indicated that he ran millions of them thru PFGW after making his most recent change on Jan. 29th with no problems found. The problem is that it is missing many of them. In many cases it is only finding a small percentage of n to be removed when all n should be removed.

3. Some of the biggest problems are the integration with existing code. The program is crashing with certain sets of algebraic factors and with certain starting lower bounds of n. It also cannot handle a file of new forms to sieve and cannot handle an already sieved file. About the only thing that it can handle are individual forms typed in at the command prompt.

Before the final program is implemented after multiple changes and testing, part of the final test plan will be to run numerous algebraic factors from algebraic.log thru PFGW to verify that they are correct. If they are properly checked in testing no more checking will be needed.

[gd_barnes]

After discussion with Mark, I have tweaked the project definition to add #4 and #5 in the "remove partial n" section as well as to remove the section titled "analysis of current situation" since it is no longer applicable.

The Aurelian algebraic factors are surprisingly varied and complex.

I believe the definition is mostly complete now but it's possible that more algebraic factors could be found in the future that we may want to add.

[pepi37]

If we look at table of reservations: not more then 20 persons and BOINC do CRUS search. Persons like me, with limited resources will remove anything they can before start LLR. For BOINC it is pretty much irrelevant, because they will finish whatever they take few days more or less.
So only way that all use data without algebraic is :
1: we make new software that will automatically remove those factors ( and it is hard process and must have long testing period)
2. for all data available for testing: to be searched, algebraic factors removed and updated "version" of data is uploaded.

So gd_barnes can you do that?

[gd_barnes]

Originally Posted by pepi37

If we look at table of reservations: not more then 20 persons and BOINC do CRUS search. Persons like me, with limited resources will remove anything they can before start LLR. For BOINC it is pretty much irrelevant, because they will finish whatever they take few days more or less.
So only way that all use data without algebraic is :
1: we make new software that will automatically remove those factors ( and it is hard process and must have long testing period)
2. for all data available for testing: to be searched, algebraic factors removed and updated "version" of data is uploaded.

So gd_barnes can you do that?

There are a lot of sieve files. My thinking has always been that the sieve files need to be looked at before people begin testing. They then need to possibly sieve more deeply and/or remove algebraic factors as needed. After all software and computing power changes constantly. What may be an optimum sieve depth today will likely be different for others or in the future.

What does not change are algebraic factors. If we can have a program such as yours that will automatically and accurately remove them for people then that is certainly helpful. That way people don't have to learn very much about them.

[pepi37]

Originally Posted by gd_barnes

There are a lot of sieve files. My thinking has always been that the sieve files need to be looked at before people begin testing. They then need to possibly sieve more deeply and/or remove algebraic factors as needed. After all software and computing power changes constantly. What may be an optimum sieve depth today will likely be different for others or in the future.

What does not change are algebraic factors. If we can have a program such as yours that will automatically and accurately remove them for people then that is certainly helpful. That way people don't have to learn very much about them.

It is not my program it is Batalov script with GUI for Windows :)
But that is irrelevant. What I need to say is that people are to lazy to do anything with data when they are reserved. So I suggest for first step to remove algebraic factors for all sieve files available for reservation. That will be huge step forward. And then it will be easy for new data ( when come) to check is any algebraic factors left ,and remove it before you upload it and when become ready for LLR-ing.
Step by step :)

[gd_barnes]

Originally Posted by pepi37

It is not my program it is Batalov script with GUI for Windows :)
But that is irrelevant. What I need to say is that people are to lazy to do anything with data when they are reserved. So I suggest for first step to remove algebraic factors for all sieve files available for reservation. That will be huge step forward. And then it will be easy for new data ( when come) to check is any algebraic factors left ,and remove it before you upload it and when become ready for LLR-ing.
Step by step :)

See my PM. We will work on putting an effort together to clean up all of the posted sieve files when more testing has been done on Batalov's script with your GUI.

[MisterBitcoin]

Originally Posted by gd_barnes

See my PM. We will work on putting an effort together to clean up all of the posted sieve files when more testing has been done on Batalov's script with your GUI.

I´m able to join this effort, too. Let me know which Bases I should take, if it´s starting.

[gd_barnes]

Originally Posted by pepi37

My friend, with my support write small program 100% based on Batalov script. I have no plan to add more function then only two have implemented: so it will check only npg files, can list algeberic factors, and can save it in factors.txt , prepared for removing with srsieve. Batalov script is still superior because it is way faster, and can check many npg files at once. First , it need more testing, but it exist.

I have done a little bit of testing on this program. Specifically I have tested it for the "remove all n" cases in the project definition. It does not handle any of them correctly. Analysis:

1. For k^2*(b^2)^n-1, it only removes even n.
2. For k*3*(b^3)^n-1 or +1, it only removes n==(0 mod 3).
3. For k*5*(b^5)^n-1 or +1, it only removes n==(0 mod 5).
4. For 4*k^4*(b^4)^n+1, it only removes n==(0 mod 4).

In all of these cases it needs to remove all n.

Obviously Batalov's script was only built for the "removal partial n" cases as shown in the project definition, namely n==(0 mod 2), n==(0 mod 3), etc. for standard forms and n==(0 mod 4) for Aurelian factors. I did not yet test any of those cases. But by observing what it did while testing the "removal all n" cases, I do think it handles #1, #2, and #3 of the "remove partial n" cases correctly but cannot say about #4 or #5 yet.

We do need to cover everything so the program is not ready for project-wide work yet. Obviously it works in cases where we know that all n do not need to be removed so it is fine for individual sieve file efforts but it needs to be modified to handle all cases correctly before we do a mass project-wide effort with it such as cleaning up all sieve files.

I will be out of town for 3 days starting on Monday. I will be back sometime Thursday. I may be on briefly in between times.