mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Msieve

Reply
 
Thread Tools
Old 2013-12-11, 21:58   #727
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

23·167 Posts
Default Additional relations from other machines

Not sure this post/query belongs here, but I'll start here, for now, since it involves factmsieve.py.

First, I have many different scripts that sieve across multiple machines using the gnfs sievers and they all seem to work fine for the most part.

However, I have recently set up a new set of scripts to work with Aliqueit across several machines, but am experiencing "error -11 reading relation ######" for what seems to be all of my externally sieved relations.

I'll describe my procedure, so that I can be told how ignorant I am and maybe among the details someone can decrease my ignorance.

When Aliqueit finishes its battery of ECM routines, which I handle via an intermediate script (different subject), and initiates factmsieve.py, I have a script that catches the fact that gnfs_<number> has been created. This script stands by for the first presence of test.job.T0 and uses it to create working polys (named numpoly) for the other machines. Effectively, I simply reuse everything except the q0 value, which I control via the script. The script then assigns the new versions of numpoly to external machines which use the following command to initiate gnfs:
Code:
gnfs-lasieve4I##e -k -a -v -o spairsM<q0 value> numpoly
## is harvested from test.log so that the correct binary is called.

I have noticed the addition of -n0/n1/n2/n3 to the command lines on the original machine, but can find no reference to their use in the documentation. For that matter, I haven't been able to find a description for the -k option, either. (My search has only been within the documentation in the ggnfs files.) Could the lack of the -n# option on my external machines be the source of my trouble?

Any thoughts/suggestions are welcome.

Thanks to all.

P.S. @Brian: I never did get mpi working within the script, but have put that on a back burner for now. The current script (above) has no modifications from your original 0.76.
EdH is offline   Reply With Quote
Old 2013-12-11, 22:33   #728
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

133778 Posts
Default

-11 means that after dividing by all the factors in the dat file and trial dividing upto 1000 the result isn't 1 for the rational polynomial.

Looking at the source for gnfs-lasieve
Code:
   while((option= getopt(argc,argv,"FJ:L:M:N:P:RS:ab:c:f:i:kn:o:q:rt:vz"))!=-1) {

      switch(option) {

      case 'R':

	g_resume = 1; break;

	case'F':

	  force_aFBcalc= 1;

	break;

	case'J':

	  NumRead(J_bits);

	break;

	case'L':

	  sysload_cmd= optarg;

	break;

	case'M':

	  if(sscanf(optarg,"%hu",&cmdline_first_mpqs_side)!=1)

	    complain("-M %s ???\n",optarg);

	break;

	case'P':

	  if(sscanf(optarg,"%hu",&cmdline_first_psp_side)!=1)

	    complain("-P %s ???\n",optarg);

	break;

	case'S':

	  if(sscanf(optarg,"%f",&sigma)!=1) {

	    errprintf("Cannot read floating point number %s\n",optarg);

	    Usage();

	  }

	break;

	case'a':

	  if(special_q_side!=NO_SIDE) {

	    errprintf("Ignoring -a\n");

	    break;

	  }

	special_q_side= ALGEBRAIC_SIDE;

	break;

	case'b':

	  if(base_name!=NULL)errprintf("Ignoring -b %s\n",base_name);

	  else base_name= optarg;

	break;

	case'c':

	  NumRead(sieve_count);

	break;

	case'f':

	  if(sscanf(optarg,"%u:%u:%u",&first_spq,&first_spq1,&first_root)!=3) {

	    if(sscanf(optarg,"%u",&first_spq) == 1) {

	      first_spq1= first_spq;

	      first_root= 0;

	    } else Usage();

	  } else append_output= 1;

	break;

	case'i':

	  if(sscanf(optarg,"%hu",&cmdline_first_sieve_side)!=1)

	    complain("-i %s ???\n",optarg);

	break;

	case'k':

	  keep_factorbase= 1;

	break;

	case'n':

	  catch_signals= 1;

	case'N':

	  NumRead(process_no);

	break;

	case'o':

	  g_ofile_name= optarg;

	break;

	case'q':

	  NumRead16(special_q_side);

	break;

	case'r':

	  if(special_q_side!=NO_SIDE) {

	    errprintf("Ignoring -r\n");

	    break;

	  }

	special_q_side= RATIONAL_SIDE;

	break;

	case't':

	  if(sscanf(optarg,"%hu",&cmdline_first_td_side)!=1)

	    complain("-t %s ???\n",optarg);

	break;

	case'v':

	  verbose++;

	break;

	case'z':

	  zip_output= 1;

	break;

      }

    }

Last fiddled with by henryzz on 2013-12-11 at 22:38
henryzz is online now   Reply With Quote
Old 2013-12-11, 23:20   #729
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

1111000000012 Posts
Default

Quote:
Originally Posted by henryzz View Post
-11 means that after dividing by all the factors in the dat file and trial dividing upto 1000 the result isn't 1 for the rational polynomial.

Looking at the source for gnfs-lasieve
Code:
   while((option= getopt(argc,argv,"FJ:L:M:N:P:RS:ab:c:f:i:kn:o:q:rt:vz"))!=-1) {

      switch(option) {

      case 'R':

    g_resume = 1; break;

    case'F':

      force_aFBcalc= 1;

    break;

    case'J':

      NumRead(J_bits);

    break;

    case'L':

      sysload_cmd= optarg;

    break;

    case'M':

      if(sscanf(optarg,"%hu",&cmdline_first_mpqs_side)!=1)

        complain("-M %s ???\n",optarg);

    break;

    case'P':

      if(sscanf(optarg,"%hu",&cmdline_first_psp_side)!=1)

        complain("-P %s ???\n",optarg);

    break;

    case'S':

      if(sscanf(optarg,"%f",&sigma)!=1) {

        errprintf("Cannot read floating point number %s\n",optarg);

        Usage();

      }

    break;

    case'a':

      if(special_q_side!=NO_SIDE) {

        errprintf("Ignoring -a\n");

        break;

      }

    special_q_side= ALGEBRAIC_SIDE;

    break;

    case'b':

      if(base_name!=NULL)errprintf("Ignoring -b %s\n",base_name);

      else base_name= optarg;

    break;

    case'c':

      NumRead(sieve_count);

    break;

    case'f':

      if(sscanf(optarg,"%u:%u:%u",&first_spq,&first_spq1,&first_root)!=3) {

        if(sscanf(optarg,"%u",&first_spq) == 1) {

          first_spq1= first_spq;

          first_root= 0;

        } else Usage();

      } else append_output= 1;

    break;

    case'i':

      if(sscanf(optarg,"%hu",&cmdline_first_sieve_side)!=1)

        complain("-i %s ???\n",optarg);

    break;

    case'k':

      keep_factorbase= 1;

    break;

    case'n':

      catch_signals= 1;

    case'N':

      NumRead(process_no);

    break;

    case'o':

      g_ofile_name= optarg;

    break;

    case'q':

      NumRead16(special_q_side);

    break;

    case'r':

      if(special_q_side!=NO_SIDE) {

        errprintf("Ignoring -r\n");

        break;

      }

    special_q_side= RATIONAL_SIDE;

    break;

    case't':

      if(sscanf(optarg,"%hu",&cmdline_first_td_side)!=1)

        complain("-t %s ???\n",optarg);

    break;

    case'v':

      verbose++;

    break;

    case'z':

      zip_output= 1;

    break;

      }

    }
Thanks henryzz - I don't know why I didn't think to look in the source code.

As for the error, I guess I'm still in the dark. I'm sieving the algebraic side and this error is showing for every relation within a huge range:
Code:
Tue Dec 10 22:52:43 2013  error -11 reading relation 520432
Tue Dec 10 22:52:43 2013  error -11 reading relation 520432
Tue Dec 10 22:52:43 2013  error -11 reading relation 520432
...
Tue Dec 10 22:53:10 2013  error -11 reading relation 1494239
Tue Dec 10 22:53:10 2013  error -11 reading relation 1494240
Tue Dec 10 22:53:10 2013  error -11 reading relation 1494241
which appears to be 973810 consecutive relations with this error.
EdH is offline   Reply With Quote
Old 2013-12-11, 23:26   #730
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

74018 Posts
Default Additional trouble

Now I'm even getting new problems!

host machine:
Code:
-> gnfs-lasieve4I12e -k -o spairs.out.T0 -v -n0 -a test.job.T0
external machine:
Code:
gnfs-lasieve4I12e -k -a -v -o spairsM<q0 value> numpoly
error from above command:
Code:
...
gnfs-lasieve4I12e: L1_BITS=15, SVN $Revision$
FBsize 96624+0 (deg 4), 183071+0 (deg 1)
xFBgen: 1 not a root mod 3
Here are the two poly files - test.job.T0:
Code:
n: 234116943738789379580780771281706880155166789502714888373927832920971823160511569007205692351556999243201
c4: 696
c3: 1877680604
c2: -6357206458268583
c1: 42147300617135555669784
c0: -81999997347928637970881448660
Y1: 9662273440429
Y0: -24082733738201157025854559
skew: 4717037.54
rlim: 2500000
alim: 1249999
lpbr: 26
lpba: 26
mfbr: 49
mfba: 49
rlambda: 2.6
alambda: 2.6
q0: 1250000
qintsize: 37500
#q1:1287500
numpoly (for external machine):
Code:
n: 234116943738789379580780771281706880155166789502714888373927832920971823160511569007205692351556999243201
c4: 696
c3: 1877680604
c2: -6357206458268583
c1: 42147300617135555669784
c0: -81999997347928637970881448660
Y1: 9662273440429
Y0: -24082733738201157025854559
skew: 4717037.54
rlim: 2500000
alim: 1249999
lpbr: 26
lpba: 26
mfbr: 49
mfba: 49
rlambda: 2.6
alambda: 2.6
q0: 2500000
qintsize: 37500
#q1:1287500
?!?

Thanks for any and all assistance...
EdH is offline   Reply With Quote
Old 2013-12-12, 14:23   #731
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

23·167 Posts
Default

OK, the new problem is fixed! I had a conflicting numpoly.afb.0 file. Maybe this was the trouble with the errors, too?
EdH is offline   Reply With Quote
Old 2013-12-12, 21:57   #732
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

23×167 Posts
Default

Ok, all appears to be working correctly now. Apparently, the problem was that I was using the same name (numpoly) for more than one instance of the siever and all the instances were writing to a single numpoly.afb.0 file which was not handling it well. I have changed the scripts to use unique numpolys and all seems well.
EdH is offline   Reply With Quote
Old 2013-12-26, 17:10   #733
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

23×167 Posts
Default MPI Modifications to Script Are Successful

@Brian: Is this a typo in line 1060 of the original version 0.76?:
Code:
    print('{0:s}, {1:d}'.format(tu[1], int(tu[0]) - int(tu[1])), FILE = outf)
@all: I have successfully modified factmsieve.py to use mpi for the msieve LA step. The modifications can probably use some cleaning up from its current amateurish version.

Here is a link to the current file: factmsievempi.py

All comments/suggestions welcome...
EdH is offline   Reply With Quote
Old 2014-04-22, 16:25   #734
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

33658 Posts
Default

I'm having some trouble with factmsieve on Ubuntu, and I think it's a stupid error, but I don't know why I'm suddenly getting it. I'm trying to factor a simple C122. I was able to use factmsieve on a c151 previously (on the exact same system) without issue. Now, however, it fails saying that readPoly() can't find a valid N.

Both the C122 and C151 fb files were created on a Windows machine, but as I said, the C151 worked fine previously. Any ideas on what I'm missing here?
wombatman is offline   Reply With Quote
Old 2014-04-22, 18:07   #735
debrouxl
 
debrouxl's Avatar
 
Sep 2009

11110100012 Posts
Default

Try changing the line endings to LF for the Linux machine.
debrouxl is offline   Reply With Quote
Old 2014-04-22, 19:53   #736
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

13×137 Posts
Default

Tried that (using Notepad++), and I get the same error. It's failing when it calls procrels. I had to compile that (and the other non-lasieve_4 64-bit files) myself since I only had the Windows binaries. Could that have something to do with it?


Edit: Seems the issue was with the .ini file. I used echo to output the number to a new ini in Terminal, and now everything is hunky-dory. Thanks for the help!

Last fiddled with by wombatman on 2014-04-22 at 20:21
wombatman is offline   Reply With Quote
Old 2014-05-30, 17:07   #737
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

13·137 Posts
Default

I'm still having the same issue as before, now with the fb file for a C122. I've copied the file onto my Ubuntu computer, opened it in GEdit, and changed the line endings to Unix/Linux. I did the same echo to file for the c122.ini.

And I still get the
Code:
readPoly(): Did not find a valid 'n' value!
What am I missing here?
wombatman is offline   Reply With Quote
Reply



Similar Threads
Thread Thread Starter Forum Replies Last Post
Msieve & ggnfs on MacOS xilman Msieve 8 2017-05-20 00:12
Factorizing with MSIEVE, GGNFS & Factmsieve.py Romuald Msieve 24 2015-11-09 20:16
Infinite loop for ggnfs or msieve Greebley Aliquot Sequences 4 2013-02-06 19:28
Error running GGNFS+msieve+factmsieve.py D. B. Staple Factoring 6 2011-06-12 22:23
A new driver? (or type of driver?) 10metreh Aliquot Sequences 3 2010-02-15 15:57

All times are UTC. The time now is 23:22.


Fri Aug 6 23:22:58 UTC 2021 up 14 days, 17:51, 1 user, load averages: 3.67, 3.95, 4.00

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.