Yafu

[RichD]

Originally Posted by schickel

What kind of number would you consider within your resources? I'm sure we could find something interesting within your reach....

I have been able to compile ECM & msieve on my PowerPC Macs. (I have several.) So anything above ~C90 would not be feasible.

I have worked many numbers (using ECM) in the C100+ range. I soon realized after several no-finds there is no place to report my work. Of course, a partial factorization would get the number within my range. :)

[schickel]

Originally Posted by RichD

I have been able to compile ECM & msieve on my PowerPC Macs. (I have several.) So anything above ~C90 would not be feasible.

I have worked many numbers (using ECM) in the C100+ range. I soon realized after several no-finds there is no place to report my work. Of course, a partial factorization would get the number within my range. :)

Hmmm, well with a target that low, there's not going to be much unless we get a downdriver run to drop something down below 100 digits.

Depending on available speed, I would actually consider msieve for anything up to ~99 digits if you're willing to wait. (You can actually split an msieve job between two or more PCs: run msieve on each machine for a while, then stop them and move the msieve.dat files to one system and CAT them together, then restart msieve.)

Another thing to consider would be to set up ECMNET on your Macs (if they're networked; though I'm not sure if rogue has a PowerPC version). That's the way I ran things before I got as far into the factoring jobs as I have: run some quick ECM with Dario's applet, put the composite into ECMNET for some intense ECM, then msieve if ECM didn't get it (I think the biggest msieve job I ever ran was 101 digits; it took several days back then).

[LaurV]

Finished nfs sieving for 996666 aliquot, T905, C105. Then yafu died, it looks like stoned, on the processing relations process. I looked on the folder, it was writing like crazy into nfs.log thousands of lines: "error -11 reading relation xxxx". I have searched the folders for files with combinations of these words. The "error %d reading relation %u" appears in yafu.exe, msieve.exe, and a couple of files from gnfs. However, not yafu nor msieve sources contain this stuff, and I suspect it comes from some #include or .dll reference into the ggnfs, for which I have no sources.

nfs.dat seems to be ok. The paths to ggnfs are ok (otherwise yafu will say immediately that "ggnfs not found, switching to siqs").

How to continue? What do I miss?

I hate nfs...

[bsquared]

Originally Posted by LaurV

Finished nfs sieving for 996666 aliquot, T905, C105. Then yafu died, it looks like stoned, on the processing relations process. I looked on the folder, it was writing like crazy into nfs.log thousands of lines: "error -11 reading relation xxxx". I have searched the folders for files with combinations of these words. The "error %d reading relation %u" appears in yafu.exe, msieve.exe, and a couple of files from gnfs. However, not yafu nor msieve sources contain this stuff, and I suspect it comes from some #include or .dll reference into the ggnfs, for which I have no sources.

nfs.dat seems to be ok. The paths to ggnfs are ok (otherwise yafu will say immediately that "ggnfs not found, switching to siqs").

How to continue? What do I miss?

I hate nfs...

The messages are coming from msieve - as it is reading the data file (nfs.dat) it is finding that the relations stored there are not relations of the corresponding polynomial (stored in the nfs.job file). So either the .job file or the .dat file are out of sync with each other, or one or both have been corrupted somehow. This could happen if another yafu process is running a different nfs job in the same folder, or if an old nfs.job file got copied in by mistake or something. Could any of that be going on?

[jasonp]

Originally Posted by LaurV

However, not yafu nor msieve sources contain this stuff

Did you check line 371 of gnfs/filter/duplicate.c, at least for the current SVN?

[LaurV]

Originally Posted by bsquared

Could any of that be going on?

Theoretically yes, if I mismatched the folders. I have a lot of "playing/testing/trying to understand" folders where I play with different things. Practically not, because it was an overnight job, so nobody to mess with it, and moreover, the nfs.job and nfs.fb contains the number to be factored as the first line, so they were not overwritten. Also, the nfs.dat file is over 900 megs, and the possibility to be a wrong one is zero (I had no big composite to factor in this computer, that would generate such a big file, for quite a long time).

I also tried to use older versions of yafu/msieve, with the same result. I even "edited" a copy of nfs.dat file and substituted all <LF> at the end of the lines with <CR><LF>, assuming this could be a problem (process that took me almost one hour, as the file is so big and all text/binary editors I use are snaily slow in handling such big file). The result is the same.

Tried to use msieve with -nc switch, newer and older version. Same result. Moved all the fullhouse (folders and files) on an XP32 computer and used the 32 bit versions. Same story.

Somehow the files don't fit each-other.

Then it pops to my mind to try a smaller nfs to see what's going on, and I tried to use yafu nfs(rsa(285)) (smaller values will automatically switch to siqs). The result is THE SAME. After crunching for some time, it come to "commencing msieve filtering" and the nfs.log file starts growing.

The fullhouse moved on the XP32 computer works ok for "yafu nfs(rsa(285))"

Now the problem is clear only on THIS computer (which is a sony laptop, core 2 at 2.8gigs, 8gig ram, Vista 64) for both yafu 32 and 64 versions. If I generate any nfs.dat file, there is no way to filter it, neither with yafu or msieve, and neither if I move all files on an XP32 and try older versions of the programs, 32 or 64 bits.

The "tune()" says (on yafu.ini):
tune_info=Intel(R) Core(TM)2 Duo CPU T9600 @ 2.80GHz, WIN32 (?!?!?), 0.000267994, 0.194719, 70.8287, 0.0725334, 102.179, 2775.7

if this is of any help. (manually added the spaces, question marks and bold face on the line above)

What I can try to do is to pass the nfs.dat and nfs.job/fb to someone who really knows what is going on, and he tries to filter it to see where it is hiccupping. There is a (heavy packed with .rar) 400 megs archieve including the 3 files (dat, job, fb). If someone wants to try filtering it to debug the problem, and to avoid losing a week of sieving, please tell me how can I pass it to you. Meantime I will run siqs on this composite. If someone tells me how to convince yafu to always run siqs, regardless of the size of the composite, no matter ho long time will take (when this is possible, I dont want to factor numbers higher than, say, C115 with siqs, but I dont like yafu switching to nfs for a C105, at all!!) then I would be deeply indebted.

Originally Posted by jasonp

Did you check line 371 of gnfs/filter/duplicate.c, at least for the current SVN?

Well, my total commander's "search in files" missed that! I did not bother to unpack all files, as I look to them directly in the zips, this process is transparent in tcmd, he treats zips like normal folders. He is able to search into the archives, any layer of deep (like zip in a zip in a zip, thousands of layers), but somehow I unchecked the checkbox for it ... It seems it got confused by a tar into a gz file and he checked only first layer. Of course, there is no text like that in the binary of the packed tar file.. However he searched well into the yafu compressed package (which is only a zip). Seems like bsquared is using your object files.

[bsquared]

Post a zip containing the nfs.job nfs.fb and a few lines copied out of nfs.dat (if this is easy for you to manage - i.e. if you have unxutils installed and can therefore 'head' out a few lines). This might tell enough without having to post the entire data set somewhere. If you can't get a few lines out of nfs.dat, don't worry about it, just post the other two.

As for making yafu always run siqs, just delete (or move) the gnfs-lasieve*.exe files. Yafu (assuming you are using the latest version) will revert to siqs if it can't find the lattice sievers.

Possibly the OS has something to do with it, but I can't think of a good reason why it would fail on vista 64 alone...

[LaurV]

Done. I used a text viewer, copy, then paste into text editor (programmers' notepad 2), it could be that the line ends were modified from LF to CRLF, but as I found before, this has no influence. The nfs64.dat files contain relations from the beginning of the criminal file, and respectively from the end of it. Please rename them accordingly, if need (delete 64xxx from the names). The nfs32.dat contains a copy/paste from a job in progress, same polinomial, but moved with all the house onto a winXP32 computer, and using yafu32 (already 150megs of relations, so the progress is about 1/8 from the size of the criminal file). They look quite differently, the last number on each line is a magnitude bigger, and as I saw by reading mseive source pointed by jason, the -11 error is somehow related to the size of the numbers. He may not like the higher-bits integers, or so?

[BudgieJane]

Another odd factorization:

>> factor(4130428535041623067988932531)

factoring 4130428535041623067988932531
using pretesting plan: normal

div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 1, starting 1000 iterations on C23
rho: x^2 + 3, starting 1000 iterations on C23
rho: x^2 + 3, starting 1000 iterations on C18
rho: x^2 + 2, starting 1000 iterations on C18
b = 1
Total factoring time = 0.0420 seconds


***factors found***

P2 = 23
P2 = 23
P3 = 367
PRP5 = 43649
C1 = 1
C18 = 487415210250370733

ans = 1

>> factor(487415210250370733)

factoring 487415210250370733
using pretesting plan: normal

div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 1, starting 1000 iterations on C18
rho: x^2 + 3, starting 1000 iterations on C18
rho: x^2 + 2, starting 1000 iterations on C18
b = 1
Total factoring time = 0.0490 seconds


***factors found***

C1 = 1
C18 = 487415210250370733

ans = 1

>>

factor.log contains

10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, ****************************
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, Starting factorization of 4130428535041623067988932531
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, ****************************
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, rho: x^2 + 1, starting 1000 iterations on C23
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, rho: x^2 + 3, starting 1000 iterations on C23
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, prp5 = 43649
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, rho: x^2 + 3, starting 1000 iterations on C18
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, rho: x^2 + 2, starting 1000 iterations on C18
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, starting smallmpqs on C18: 487415210250370733
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, prp1 = 1
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, prp18 = 487415210250370733
10/07/11 10:55:13 v1.28.5 @ JANELAPTOP2, Total factoring time = 0.0420 seconds
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, 
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, ****************************
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, Starting factorization of 487415210250370733
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, ****************************
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, rho: x^2 + 1, starting 1000 iterations on C18
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, rho: x^2 + 3, starting 1000 iterations on C18
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, rho: x^2 + 2, starting 1000 iterations on C18
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, starting smallmpqs on C18: 487415210250370733
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, prp1 = 1
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, prp18 = 487415210250370733
10/07/11 10:55:36 v1.28.5 @ JANELAPTOP2, Total factoring time = 0.0490 seconds

The smaller number factors normally if I multiply it by 2147483647:

>> factor(487415210250370733*2147483647)

factoring 1046716193311737924804903251
using pretesting plan: normal

div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 1, starting 1000 iterations on C28
rho: x^2 + 3, starting 1000 iterations on C28
rho: x^2 + 2, starting 1000 iterations on C28
b = 1
Total factoring time = 0.1000 seconds


***factors found***

PRP9 = 837532169
PRP9 = 581965957
PRP10 = 2147483647

ans = 1

[schickel]

Excerpt from a quick test on my new hex-core:

HEXADECIMAL, starting SIQS on c101: 12779726826463409629092032652842999953570478140774811249429486795957571836464080419971376622939202021
HEXADECIMAL, ==== sieving started ( 6 threads) ====
HEXADECIMAL, recovered 14 nontrivial dependencies
HEXADECIMAL, prp42 = 868090230181216249188964028974950768080011
HEXADECIMAL, prp59 = 14721657245002752906581031455951927854123538531478714392911
HEXADECIMAL, Lanczos elapsed time = 92.8360 seconds.
HEXADECIMAL, Sqrt elapsed time = 0.4050 seconds.
HEXADECIMAL, SIQS elapsed time = 2715.8820 seconds.

[bsquared]

Originally Posted by schickel

Excerpt from a quick test on my new hex-core:

HEXADECIMAL, starting SIQS on c101: 12779726826463409629092032652842999953570478140774811249429486795957571836464080419971376622939202021
HEXADECIMAL, ==== sieving started ( 6 threads) ====
HEXADECIMAL, recovered 14 nontrivial dependencies
HEXADECIMAL, prp42 = 868090230181216249188964028974950768080011
HEXADECIMAL, prp59 = 14721657245002752906581031455951927854123538531478714392911
HEXADECIMAL, Lanczos elapsed time = 92.8360 seconds.
HEXADECIMAL, Sqrt elapsed time = 0.4050 seconds.
HEXADECIMAL, SIQS elapsed time = 2715.8820 seconds.

Nice!