Resume dependency processing?

[sean]

I'm trying to factor the C195 cofactor of Pell(583) by SNFS using Yafu. All the sieving completed, linear algebra completed, and it produced a 51-digit prime factor from dependency 2, but left a C145:

nfs: commencing msieve filtering
555109312683062119393323568281537098290442051919826633599191946398743120653033966337716794848068138895876548755215361212319751875346944966744570403598148867381515843660226447009202609523433561321
read 10M relations
read 20M relations
read 30M relations
read 40M relations
read 50M relations
nfs: commencing msieve linear algebra
linear algebra completed 5902018 of 5902337 dimensions (100.0%, ETA 0h 0m)
nfs: commencing msieve sqrt
NFS elapsed time = 524670.8332 seconds.

***factors found***

P51 = 336246287794168748415846490876768360738264937817769
C145 = 1650900940274080070056842767142908087541932084478741439796300957019264037849654528604010200642820006573008520577140256589131790152238203175660609

I expected yafu to continue on with other dependencies and (hopefully) factor the C145, but that didn't happen. Is there some way that I can tell it to restart with the next dependency?

Fri Oct 13 20:45:07 2017  commencing square root phase
Fri Oct 13 20:45:07 2017  reading relations for dependency 1
Fri Oct 13 20:45:08 2017  read 2950961 cycles
Fri Oct 13 20:45:12 2017  cycles contain 8715406 unique relations
Fri Oct 13 20:45:49 2017  read 8715406 relations
Fri Oct 13 20:46:33 2017  multiplying 8715406 relations
Fri Oct 13 20:51:29 2017  multiply complete, coefficients have about 222.00 million bits
Fri Oct 13 20:51:30 2017  initial square root is modulo 92679589
Fri Oct 13 20:57:12 2017  GCD is 1, no factor found
Fri Oct 13 20:57:12 2017  reading relations for dependency 2
Fri Oct 13 20:57:13 2017  read 2950151 cycles
Fri Oct 13 20:57:17 2017  cycles contain 8716322 unique relations
Fri Oct 13 20:57:54 2017  read 8716322 relations
Fri Oct 13 20:58:37 2017  multiplying 8716322 relations
Fri Oct 13 21:04:08 2017  multiply complete, coefficients have about 222.03 million bits
Fri Oct 13 21:04:10 2017  initial square root is modulo 92925121
Fri Oct 13 21:10:56 2017  sqrtTime: 1549

[VictordeHolland]

From the MSIEVE help "-h"
try the "-ncr" command to tell msieve to resume Linear Algebra from a checkpoint
or
"-nc3" to only perform NFS square root

Square root options:
dep_first=x
dep_last=y
with x, y between 1 and 64

[bsquared]

Originally Posted by VictordeHolland

From the MSIEVE help "-h"
try the "-ncr" command to tell msieve to resume Linear Algebra from a checkpoint
or
"-nc3" to only perform NFS square root

Square root options:
dep_first=x
dep_last=y
with x, y between 1 and 64

You'd want the -nc3 option. However it looks like I didn't plan for this super well:

else if (strcmp(opt,OptionArray[56]) == 0)
	{
		//argument "nc3".  do nfs sqrt
		fobj->nfs_obj.nfs_phases |= NFS_PHASE_SQRT;
	}

which means yafu only recognizes -nc3 as "do nfs sqrt"; it doesn't let you choose the dependency to try. So the best method from here is probably to use msieve directly. I think (backup your files first!) you could change the names of the data files (matrix file, data file, dependency file) to msieve.* and then start msieve with <your number> -nc3 "dep_first=x, dep_last=y"

[sean]

Originally Posted by bsquared

I think (backup your files first!) you could change the names of the data files (matrix file, data file, dependency file) to msieve.* and then start msieve with <your number> -nc3 "dep_first=x, dep_last=y"

Thanks for this suggestion. I haven't used msieve directly much. The files I have left over from yafu (other than various logs) are just:

-rw-rw-r-- 1 sean rtg        463 Sep 19 22:23 nfs.job
-rw-r--r-- 1 sean rtg        308 Oct  7 19:26 yafu.ini
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 19:04 nfs.dat.bak.chk
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 20:04 nfs.dat.chk

The original relations file, nfs.dat, file appears to have been deleted as part of the linear algebra and perhaps at the conclusion of the run? Will that even be enough files to resume from?

I tried copying nfs.dat.chk to msieve.dat.chk and nfs.job to msieve.job and running msieve. But it does not find the polynomial when I try to start msieve:

msieve -nc3 3,5 555109312683062119393323568281537098290442051919826633599191946398743120653033966337716794848068138895876548755215361212319751875346944966744570403598148867381515843660226447009202609523433561321
error generating or reading NFS polynomials

I don't know the file name conventions for msieve. I've looked through the help, but it is not very clear about what file names it is looking for.

[VictordeHolland]

Originally Posted by sean

I don't know the file name conventions for msieve. I've looked through the help, but it is not very clear about what file names it is looking for.

I usually have something like this after a msieve run:
C204_119_99.dat.gz (compressed relations from NFS@home)
msieve.dat (uncompressed relations)
msieve.dat.bak.chk (backup checkpoint)
msieve.dat.chk (checkpoint)
msieve.dat.cyc
msieve.dat.dep
msieve.dat.mat (matrix)
msieve.fb
msieve.poly (poly)
msieve.projection (you dont need this file, its from NFS@home I believ)
worktodo.ini (the to factor number goes in here)
+
some .dlls the .exe readme/changes since its a Windozzzz system

[Dubslow]

Originally Posted by sean

Thanks for this suggestion. I haven't used msieve directly much. The files I have left over from yafu (other than various logs) are just:

-rw-rw-r-- 1 sean rtg        463 Sep 19 22:23 nfs.job
-rw-r--r-- 1 sean rtg        308 Oct  7 19:26 yafu.ini
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 19:04 nfs.dat.bak.chk
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 20:04 nfs.dat.chk

The original relations file, nfs.dat, file appears to have been deleted as part of the linear algebra and perhaps at the conclusion of the run? Will that even be enough files to resume from?

I tried copying nfs.dat.chk to msieve.dat.chk and nfs.job to msieve.job and running msieve. But it does not find the polynomial when I try to start msieve:

msieve -nc3 3,5 555109312683062119393323568281537098290442051919826633599191946398743120653033966337716794848068138895876548755215361212319751875346944966744570403598148867381515843660226447009202609523433561321
error generating or reading NFS polynomials

I don't know the file name conventions for msieve. I've looked through the help, but it is not very clear about what file names it is looking for.

nfs.job is the polynomial file that msieve is looking for.

However this is moot, as the "nfs.dat" (no trailing filename tags) relations file is indeed required for all steps. Linear algebra determines how to combine the relations into a congruence of squares, where the "sqrt" performs the actual combination found previously, and then deduces the factors. But in order to perform the combination of relations specified by the solved matrix, you do in fact need the relations in their entirety. If you cannot find the entire relations file somewhere, you will need to restart from scratch for the C145, unfortunately. Definitely not a fun bug in yafu.

[bsquared]

Originally Posted by sean

Thanks for this suggestion. I haven't used msieve directly much. The files I have left over from yafu (other than various logs) are just:

-rw-rw-r-- 1 sean rtg        463 Sep 19 22:23 nfs.job
-rw-r--r-- 1 sean rtg        308 Oct  7 19:26 yafu.ini
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 19:04 nfs.dat.bak.chk
-rw-rw-r-- 1 sean rtg  236097336 Oct 13 20:04 nfs.dat.chk

The original relations file, nfs.dat, file appears to have been deleted as part of the linear algebra and perhaps at the conclusion of the run? Will that even be enough files to resume from?

Crap. No, that's not enough. As Dubslow mentioned, you need the relations file at a minimum, and certainly the dependency file too. I feel really bad about this - my apologies. I will try to get a fix uploaded soon. Probably your best option now would be to do gnfs on the c145. I can help, if you want.

[sean]

Originally Posted by bsquared

Crap. No, that's not enough. As Dubslow mentioned, you need the relations file at a minimum, and certainly the dependency file too. I feel really bad about this - my apologies. I will try to get a fix uploaded soon. Probably your best option now would be to do gnfs on the c145. I can help, if you want.

Ok. Thanks for all the comments. I can complete the C145 by GNFS easy enough. Its the first time I've actually had a problem like this happen with yafu, and on the whole I find it reliable.