mersenneforum.org Reservations for x^y+y^x
 Register FAQ Search Today's Posts Mark Forums Read

2020-10-01, 21:15   #23
pxp

Sep 2010
Weston, Ontario

26·3 Posts

Quote:
 Originally Posted by pxp I am seeing much earlier ETC dates on restarted interruptions.
Unfortunately, this fix is no longer working well on my now much larger input files. The interruptions (after about a day of sieving) run into an error that has one or more workers not responding within a ten-minute default and xyyxsieve aborts the writing of the final output file. The existing output file is smaller than the input file but I have been afraid to use it because I fear that it has been contaminated by that final aborted write attempt.

One of the problems here was that I was not letting it run long enough. The existing output file might have had "// Sieved to 5" which confused me because the run was at p=600000+. A couple of times I even had a "// Sieved to 0" which strikes me as nonsensical. Clearly this is all about when exactly the program chooses to write an output and how long it actually takes to write that output. And I understand now that it is the writing of that output for very large files that is driving the ETC dates unrealistically into the distant future.

An unfinished example is a run on interval #18's almost 28 million term input file (406 MB) started on September 26:

Code:
./xyyxsieve -i18.ini -o18.txt -W4 -p3 -P1e7
which, as I write, is at:

Code:
p=4798627, 2.361 p/sec, 24100965 factors found at 0.000 f/sec, 48.0% done. ETC 2020-10-07 07:04
The current output file is sieved to 1481 and is down to 57 MB. When I get to the end I think that I will still run into this ten-minute unresponsive-worker abort of the final output update but the existing output file at that time should be sieved to a large enough number to be usable. Should I worry that such a file may have been corrupted by that final unsuccessful write attempt? What if the aborted file-write caused the file that is there at the finish to be missing a whole bunch of terms? That would directly impact the possibility of my not finding all PRPs in that interval.

 2020-10-01, 22:46 #24 rogue     "Mark" Apr 2003 Between here and the 5×1,249 Posts The program will update the output file once every 60 minutes. The "Sieved to" value in that file is the largest prime that the program can conclusively state has been successfully sieved to with no gaps. My recommendation is to sieve with a single thread to 1e4 or 1e5 to knock out most of the candidates. Then start again on the output file with multiple threads. You should also consider using -w1e4 to reduce the number of primes per check. This also means that ^C will terminate the program faster. It might still take many minutes due to the number of terms, but it will terminate. The ETC dates are far into the future because the time to sieve the first chunk takes a really long time and the program extrapolates based upon that. The ETC will come closer to the present as the number of terms is reduced. That is why I suggest sieving to 1e4 or 1e5 with a single worker before running again with multiple threads.
2020-10-03, 19:04   #25
pxp

Sep 2010
Weston, Ontario

26×3 Posts

Quote:
 Originally Posted by rogue You should also consider using -w1e4 to reduce the number of primes per check. This also means that ^C will terminate the program faster. It might still take many minutes due to the number of terms, but it will terminate.
According to the xyyxsieve help file the default for -w is 1e4, so that is what I have (unknowingly) been using. Trying 1e3 is "out of range". What is the point of larger -w arguments? I will guess faster processing but perhaps at the expense of RAM. That is not for me an issue as I have 32 GB on all of my machines and I have not found either xyyxsieve or pfgw take any advantage of it.

 2020-10-03, 21:19 #26 pxp     Sep 2010 Weston, Ontario 3008 Posts I tried: Code: mm4:rogue1 pxp\$ ./xyyxsieve -i19.ini -o19.txt -w1e4 -p3 -P1e4 xyyxsieve v1.5, a program to find factors numbers of the form x^y+y^x Sieve started: 3 < p < 1e4 with 17782120 terms (27440 <= x <= 425055, 2 <= y <= 28701) (expecting 15661063 factors) p=0, 0.000 p/sec, 3187205 factors found at 176.5 f/sec, 0.0% done. 1 workers didn't stop after 10 minutes So the single thread and -P1e4 made no difference.
 2020-10-03, 21:43 #27 pxp     Sep 2010 Weston, Ontario 26·3 Posts What I am going to do is multi-thread these runs to -P1e9 and after several days just rescue (move elsewhere) whatever output file is there and then kill the run. Also, in the future, I think I will do some small sieving in Mathematica prior to creating the ABC files. I had been reluctant to do this initially because I figured xyyxsieve would be just so much faster.
 2020-10-03, 22:01 #28 rogue     "Mark" Apr 2003 Between here and the 5·1,249 Posts With so many candidates, it will take a long time to get to 1e4. Yes -w must be at least 1e4. The version of xyyxsieve that I gave to you won't benefit when adding more candidates. You might as well divide the input to smaller files and sieve them.
2020-10-04, 23:10   #29
pxp

Sep 2010
Weston, Ontario

26·3 Posts

Quote:
 Originally Posted by rogue The program will update the output file once every 60 minutes.
I am looking at this now. One of my runs was updated at 2:48 pm today: sieved to 11 (149.2 MB); then at 3:49 pm: sieved to 13 (142.2 MB); then at 4:50 pm: sieved to 13 (140.3 MB); then at 5:51 pm: sieved to 17 (136.3 MB); then at 6:52 pm: sieved to 17 (132.7 MB).

At this point xyyxsieve has:

Code:
Sieve started: 3 < p < 1e9 with 22133879 terms (30455 <= x <= 476849, 2 <= y <= 31873) (expecting 20960485 factors)
p=243011, 0.098 p/sec, 13031312 factors found at 5.358 f/sec, 0.0% done. ETC 2032-08-29 10:30
Note that p is past 243000. It has already found 13 million of an expected 21 million factors. Checking the 6:52 pm file, has it just over 9 million terms in it which is exactly in line with the 13 million factors found. Most troubling are the sieved-to numbers: 11, 13, 13, 17, 17. All the while p has been at 200000+. My guess is that the sieved-to numbers are incorrect. Should they not be where p is at?

Last fiddled with by pxp on 2020-10-04 at 23:15 Reason: factual correction

2020-10-05, 02:11   #30
rogue

"Mark"
Apr 2003
Between here and the

141458 Posts

Quote:
 Originally Posted by pxp Most troubling are the sieved-to numbers: 11, 13, 13, 17, 17. All the while p has been at 200000+. My guess is that the sieved-to numbers are incorrect. Should they not be where p is at?
I don't understand.

 2020-10-05, 16:22 #31 pxp     Sep 2010 Weston, Ontario 26×3 Posts When p=3 and one has divided all initial candidates by 3, one has sieved to 3. When p=5 and one has divided all remaining candidates by 5, one has sieved to 5. When p=7 and one has divided all remaining candidates by 7, one has sieved to 7. Is that not correct or the way the program works?
2020-10-05, 18:34   #32
rogue

"Mark"
Apr 2003
Between here and the

5·1,249 Posts

Quote:
 Originally Posted by pxp When p=3 and one has divided all initial candidates by 3, one has sieved to 3. When p=5 and one has divided all remaining candidates by 5, one has sieved to 5. When p=7 and one has divided all remaining candidates by 7, one has sieved to 7. Is that not correct or the way the program works?
That is correct. I did not understand this:

Quote:
 Most troubling are the sieved-to numbers: 11, 13, 13, 17, 17. All the while p has been at 200000+. My guess is that the sieved-to numbers are incorrect. Should they not be where p is at?
Are you suggesting that many p were skipped (in other words the value for "sieved to" is wrong)? Are you suggesting that some factors were missed for p that were tested? Is there some other issue? I just don't understand your comments.

 2020-10-06, 20:24 #33 pxp     Sep 2010 Weston, Ontario 19210 Posts As per my examples, when p = n has been reached, I expect the output file to be sieved to n (and you have said that my expectation is correct). Instead, the program is saying p ~ 200000 but the output file is not sieved to ~200000 but, rather, to 17. An hour later, p (supposedly, if the program value is to be believed) has advanced by thousands but the new output sieved-to number is still 17. There is no correspondence, so yes, either the sieved-to-number is wrong or the "p=" number in the console is wrong. I can't fathom either possibility which is why I had rather expected you to say that my understanding of the correspondence was incorrect.

 Similar Threads Thread Thread Starter Forum Replies Last Post ET_ Operazione Doppi Mersennes 495 2020-12-19 19:41 lukerichards And now for something completely different 13 2019-05-11 23:27 kar_bon Riesel Prime Data Collecting (k*2^n-1) 129 2016-09-05 09:23 R.D. Silverman NFS@Home 15 2015-11-29 23:18 paulunderwood 3*2^n-1 Search 15 2008-06-08 03:29

All times are UTC. The time now is 18:52.

Wed Mar 3 18:52:32 UTC 2021 up 90 days, 15:03, 2 users, load averages: 1.95, 1.71, 1.67