[QUOTE=swellman;398660]Sorry I didn't expand - this seems to be the same bug behavior I reported in [URL="http://www.mersenneforum.org/showpost.php?p=385248&postcount=300"]this post[/URL]. Lots of polys scroll by, then yafu crashes at the final "m=..." line of the last polynomial. It does not seem to attempt to evaluate the polynomials nor start the test sieving routine.

A generic windows error message pops up from the OS, saying "yafu.exe has encountered a problem and needs to close. We are sorry for the inconvenience."

The error report itself is equally unhelpful. "Error signature AppName: yafu.exe AppVer: 0.0.0.0 ModName: yafu.exe
ModVer: 0.0.0.0 Offset: 000deccc".


Can anyone else confirm this crash in Windows?[/QUOTE]


Confirmed crash occurs in Yafu 1.34 (but not in Yafu 1.33).
Attached copy of screen output.

[url]http://mersenneforum.org/showthread.php?p=400532#post400532[/url]

A couple of different problems appearing in the same command.

Any advice?

[QUOTE=Dubslow;400539][url]http://mersenneforum.org/showthread.php?p=400532#post400532[/url]

A couple of different problems appearing in the same command.

Any advice?[/QUOTE]

Looking at the messages being printed (no data file, no dependencies, no polynomials, etc...) I'd say you'll have to resolve the matrix - maybe restart from filtering. I'm not sure because I can't see what files you have exactly.

As for why you are in this boat in the first place - anytime something abnormally shuts down yafu all bets are off. Detaching the process seems to qualify.

The garbage coming out of the sqrt is likely just a consequence of the garbage coming in.

Nah it was working fine for weeks, detaching the process is external to the process, it can't tell the difference.

The command that was originally run was [code]nice -n 19 ./yafu "snfs((621011411269^17-1), (621011411269^17-1)/621011411268)" -o fuck4[/code] (This is the 5th job I ran after losing two days of work by overwriting the previous job's data after it didn't finish correctly... hence the -o option.)

I later had to resume it (with R=1 in yafu.ini).

These are the current files, followed by the command I've tried to run to redo the square root (again with R=1 in yafu.ini). This job took a couple of weeks. Yesterday I wasn't sure which files use the -o name and which (if any) keep the default "nfs" prefix when -o is specified, which is why some of those have a edit date of yesterday. nfs.job and fuck4.job are identical, each containing the C189 in question.

[code]bill@Gravemind⌚1131 ~/yafu ∰∂ ls -l
total 27172676
drwxr-xr-x 11 bill bill 4096 Sep 30 2014 dev
-rw-r--r-- 1 bill bill 34648 Feb 25 10:54 docfile.txt
-rw-r--r-- 1 bill bill 676 Feb 25 10:54 ecmhits.opn
drwxr-xr-x 2 bill bill 4096 Mar 2 00:06 eucmac
-rw-r--r-- 1 bill bill 4750957 Apr 20 18:57 factor.log
-rw-r--r-- 1 bill bill 1003 Mar 10 15:14 factors.opn
-rw-r--r-- 1 bill bill 118517536 Mar 4 09:05 fuck1.bak.chk
-rw-r--r-- 1 bill bill 118517536 Mar 4 10:04 fuck1.chk
-rw-r--r-- 1 bill bill 11192 Mar 4 10:49 fuck1.log
-rw-r--r-- 1 bill bill 97022016 Mar 7 11:51 fuck2.bak.chk
-rw-r--r-- 1 bill bill 97022016 Mar 7 13:00 fuck2.chk
-rw-r--r-- 1 bill bill 9929 Mar 7 14:41 fuck2.log
-rw-r--r-- 1 bill bill 102038936 Mar 10 11:11 fuck3.bak.chk
-rw-r--r-- 1 bill bill 102038936 Mar 10 12:13 fuck3.chk
-rw-r--r-- 1 bill bill 11782 Mar 10 14:03 fuck3.log
-rw-r--r-- 1 bill bill 264577576 Apr 17 10:47 fuck4.bak.chk
-rw-r--r-- 1 bill bill 264577576 Apr 17 11:36 fuck4.chk
-rw-r--r-- 1 bill bill 6474999437 Apr 20 18:52 fuck4.dat
-rw-r--r-- 1 bill bill 648 Apr 20 18:54 fuck4.job
-rw-r--r-- 1 bill bill 10503 Apr 20 18:57 fuck4.log
-rw-r--r-- 1 bill bill 6474999437 Mar 31 03:18 fuck4.save
-rw-r--r-- 1 bill bill 93655656 Feb 28 10:26 fuck.bak.chk
-rw-r--r-- 1 bill bill 93655656 Feb 28 11:32 fuck.chk
-rw-r--r-- 1 bill bill 9147 Feb 28 12:38 fuck.log
-rw-r--r-- 1 bill bill 225719 Apr 20 18:45 ggnfs.log
-rwxr-xr-x 1 bill bill 1458777 Feb 25 10:54 gnfs-lasieve4I11e
-rwxr-xr-x 1 bill bill 1425529 Feb 25 10:54 gnfs-lasieve4I12e
-rwxr-xr-x 1 bill bill 1411255 Feb 25 10:54 gnfs-lasieve4I13e
-rwxr-xr-x 1 bill bill 1402755 Feb 25 10:54 gnfs-lasieve4I14e
-rwxr-xr-x 1 bill bill 1396937 Feb 25 10:54 gnfs-lasieve4I15e
-rwxr-xr-x 1 bill bill 1395287 Feb 25 10:54 gnfs-lasieve4I16e
drwxr-xr-x 2 bill bill 4096 Feb 11 22:43 m991
-rw-r--r-- 1 bill bill 3705141792 Apr 20 18:45 nfs.dat
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.0.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.1.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.2.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.3.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.4.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.5.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.6.p
-rw-r--r-- 1 bill bill 0 Feb 25 10:56 nfs.dat.7.p
-rw-r--r-- 1 bill bill 93655656 Feb 25 10:56 nfs.dat.bak.chk
-rw-r--r-- 1 bill bill 93655656 Feb 25 10:56 nfs.dat.chk
-rw-r--r-- 1 bill bill 60978516 Feb 25 10:56 nfs.dat.cyc
-rw-r--r-- 1 bill bill 18730360 Feb 25 10:56 nfs.dat.dep
-rw-r--r-- 1 bill bill 172948996 Feb 25 13:34 nfs.dat.lp
-rw-r--r-- 1 bill bill 675824564 Feb 25 10:57 nfs.dat.mat
-rw-r--r-- 1 bill bill 259 Feb 25 10:57 nfs.fb
-rw-r--r-- 1 bill bill 648 Mar 10 15:29 nfs.job
-rw-r--r-- 1 bill bill 2138884579 Feb 25 13:34 nfs.log
-rw-r--r-- 1 bill bill 187 Feb 25 10:57 num
-rw-r--r-- 1 bill bill 338 Feb 25 10:57 opnwork.bat
-rwxr-xr-x 1 bill bill 944 Feb 25 10:57 run.sh
-rw-r--r-- 1 bill bill 64171 Apr 20 18:57 session.log
-rw-r--r-- 1 bill bill 67743362 Feb 25 10:57 siqs.dat
-rw-r--r-- 1 bill bill 6475000630 Mar 31 03:21 SUPERBUP
-rw-r--r-- 1 bill bill 9 Mar 31 00:59 tmp
-rw-r--r-- 1 bill bill 405 Feb 25 10:57 tune.job
-rw-r--r-- 1 bill bill 3806487 Feb 25 10:57 tunerels.out
-rwxr-xr-x 1 bill bill 1051 Mar 10 15:19 work.bat
-rwxr-xr-x 1 bill bill 2976576 Feb 25 10:57 yafu
-rw-r--r-- 1 bill bill 469 Mar 10 15:21 YAFU_get_poly_score.out
-rw-r--r-- 1 bill bill 191 Feb 28 11:59 yafu.ini
bill@Gravemind⌚1133 ~/yafu ∰∂ nice -n 19 ./yafu "snfs(621011411269^17-1, 489320362321398184677729630934943053523351985788653028535881731445833854004889310101379710244867513065593115513178000509225608472400366336771242634777571443042484105575262648351686921131441)" -o fuck4 -nc3


04/21/15 11:33:18 v1.34.5 @ Gravemind, System/Build Info:
Using GMP-ECM 7.0-dev, Powered by GMP 6.0.0
detected Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
measured cpu frequency ~= 3392.308390
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> nfs: checking for job file - job file found, testing for matching input
nfs: number in job file matches input
nfs: checking for data file
nfs: no data file found
nfs: commencing nfs on c189: 489320362321398184677729630934943053523351985788653028535881731445833854004889310101379710244867513065593115513178000509225608472400366336771242634777571443042484105575262648351686921131441
nfs: resuming with sqrt
nfs: commencing msieve sqrt

commencing square root phase
polynomials not found
sqrtTime: 0
nfs: failed to find factors... possibly no dependencies found
nfs: continuing with sieving


***factors found***

C12 = 621011411268

***co-factor***
C177 = 787941015966661508573574901085392422897792479455815738692961172457618914487623547095653286833456321455946356777950376330097592386968573073552854652883997321700700316695911668968

ans = 489320362321398184677729630934943053523351985788653028535881731445833854004889310101379710244867513065593115513178000509225608472400366336771242634777571443042484105575262648351686921131441

bill@Gravemind⌚1136 ~/yafu ∰∂ cat fuck4.job
n: 489320362321398184677729630934943053523351985788653028535881731445833854004889310101379710244867513065593115513178000509225608472400366336771242634777571443042484105575262648351686921131441
# 621011411269^17-1, difficulty: 212.28, anorm: 4.10e+37, rnorm: 2.49e+40
# scaled difficulty: 212.28, suggest sieving algebraic side
# siever: 15
# size = 9.329e-12, alpha = 0.000, combined = 6.139e-13, rroots = 2
type: snfs
size: 212
skew: 92.3669
c6: 1
c0: -621011411269
Y1: -1
Y0: 239496263202161156076544764400578109
m: 239496263202161156076544764400578109

rlim: 22600000
alim: 22600000
lpbr: 30
lpba: 30
mfbr: 60
mfba: 60
rlambda: 2.6
alambda: 2.6
[/code]

Notice that the previous jobs (fuck2, fuck3 etc) seem to be missing their data file... there was a fuck4.save, which I renamed to fuck4.dat. Edit: Upon reviewing the ls output, that's a file I made (along with SUPERBUP) when I was fiddling with the data on March 31st as part of the resuming process. So it would seem that the relations are in fact lost. Why isn't the data file kept and/or named properly after the -o option?

First of all, I [I]highly highly[/I] recommend putting each new job into a separate folder. Are each of these f-jobs different jobs or previous attempts at this job?? Its very hard for me to sort out what's going on.

If those are previous jobs then all appears well with them. Yafu is supposed to clean up after itself after a successful factorization, which it appears to have done for f-job 1,2, etc. (Note: .chk files are not cleaned up because I think jason implemented that feature after I wrote the cleanup routine and I never went back and updated it.)

f-job4 appears to be missing .dep and .mat, which I think, (someone correct me if I'm wrong) are necessary for the sqrt stage. Are the various nfs.* files associated with f-job4?? I'm still pretty confused.

Yes, I'd recommend naming everything associated with f-job4 as nfs.*. The NFS-restart code is far from bulletproof and it's entirely likely that naming issues could cause problems.

[Edit after reading edits]
Based on all of this I would stop using -o entirely. From now on if you still want to use yafu - make separate folders and always use the default naming.

They're all separate jobs, I was trying to avoid the hassle of extra directories lying around. How can I tell yafu to leave the data file? AFAIK moving to a separate directory won't solve that issue

[QUOTE=Dubslow;400574]They're all separate jobs, I was trying to avoid the hassle of extra directories lying around. How can I tell yafu to leave the data file? AFAIK moving to a separate directory won't solve that issue[/QUOTE]

I guess you'll have to gauge for yourself whether creating directories is more or less of a hassle than what you've gone through lately :smile:.

I don't believe there's an option to tell yafu to leave the data file. A script that runs the stages separately, backing up the data file prior to running any postprocessing, wouldn't be that hard to come up with. But at that point you might as well use factmsieve or something.

Ooooh, gotta love necroposting.

[QUOTE=Dubslow;434429]Yafu has a [c]-nt[/c] option which *should* automatically trial sieve a list of poly files (including parameter generation, if necessary, and even parameter adjustment in rare circumstances), but last time I tried it, it was rather buggy. (Great testament to my programming abilities, since I wrote it! :smile:)[/QUOTE]

In actuality, I can only recall one major bug from years ago, which is that -nt would delete the poly files passed to it to be tested, and I was just reading through the code right now (good god I was a moron back then, sorry to pollute your program with my coding "style" crap Ben), and I think I found and fixed the bug. Only... in attempting to test it, I seem to be violating the apparent assumption that arguments to options fit in 79 bytes:

[code]run "nfs($(cat num))" -nt hybrid1.job,hybrid2.job,hybrid3.job,cpucado1.job,cpucado2.job,cpucado3.job,gpumsieve1.job,gpumsieve2.job,gpumsieve3.job -v -threads 8
Starting program: /home/bill/yafu/yafu "nfs($(cat num))" -nt hybrid1.job,hybrid2.job,hybrid3.job,cpucado1.job,cpucado2.job,cpucado3.job,gpumsieve1.job,gpumsieve2.job,gpumsieve3.job -v -threads 8
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
invalid option eve1.job,gpumsieve2.job,gpumsieve3.job
[Inferior 1 (process 16215) exited with code 01][/code]

Note the so-called "invalid option" starts with the 82nd character of the argument passed. This diagnosis is corroborated by this code in driver.c:

[code]unsigned process_flags(int argc, char **argv, fact_obj_t *fobj, char *expression)
{
int ch = 0, i,j,valid;
char optbuf[MAXOPTIONLEN];
[B]char argbuf[80];
[/B]
expression[0] = '\0';

//argument loop
i = 0;
while (i < argc)
{
//read in the option
ch = argv[i][0];
...

strcpy(optbuf,argv[i]);
...

//check to see if this option requires an argument
if (needsArg[j] == 1)
{
i++;
if ((i == argc) || (argv[i][0] == '-'))
{
printf("argument expected for %s\n",optbuf);
exit(1);
}
[B]strcpy(argbuf,argv[i])[/B];

//now apply -option argument
//printf("applying option %s with argument %s\n",optbuf+1,argbuf);
applyOpt(optbuf+1,argbuf,fobj);
}
...[/code]

Anyways, changing it to GSTR_MAXSIZE makes it work, and I just found and fixed a second bug in trial sieving, patch incoming shortly

Okay, here's the patch. Note the old "revision" listed, I haven't bothered to make sense of subversion in a while. It may or may not cleanly apply to what you have, but even if not it's all of 10 lines of diff, should be trivial to merge. Also note the TODO that really should be considered adding, but I didn't really feel like digging into the code I don't remember as well to figure out how to accomplish it (at least not at this time).

[code]Index: factor/nfs/nfs.c
===================================================================
--- factor/nfs/nfs.c (revision 345)
+++ factor/nfs/nfs.c (working copy)
@@ -1143,11 +1143,13 @@
while((ptr = strchr(arg, ','))) // this sort of thing is what's absolutely brilliant about C
{
filenames[i] = (char*)malloc(GSTR_MAXSIZE*sizeof(char));
+ memset(filenames[i], 0, GSTR_MAXSIZE);
//printf("pointer: %p\n", filenames[i]);
- strncpy(filenames[i++], arg, ptr-arg+1);
+ strncpy(filenames[i++], arg, ptr-arg);
arg = ptr + 1;
}
filenames[i] = (char*)malloc(GSTR_MAXSIZE*sizeof(char));
+ memset(filenames[i], 0, GSTR_MAXSIZE);
strcpy(filenames[i++], arg);

me = test_sieve(fobj, filenames, i, 1);
Index: factor/nfs/nfs_sieving.c
===================================================================
--- factor/nfs/nfs_sieving.c (revision 345)
+++ factor/nfs/nfs_sieving.c (working copy)
@@ -119,7 +119,7 @@
printf("malloc failed\n");
exit(-1);
}
- sprintf(filenames[i], "test-sieve-%d.poly", i);
+ sprintf(filenames[i], "snfs-test-sieve-%d.poly", i);
out = fopen(filenames[i], "w");
if( !out )
{
@@ -147,6 +147,9 @@
char syscmd[GSTR_MAXSIZE], tmpbuf[GSTR_MAXSIZE], side[32];
FILE* in;

+ if (VFLAG > 0)
+ printf("\ntest: trial sieving %s\n", filenames[i]);
+
// should probably scale the range of special-q to test based
// on input difficulty, but not sure how to do that easily...

@@ -392,7 +395,7 @@
}
else
{
- if (VFLAG > 0) printf("test: estimated total sieving time = %s (with %d threads)\n",
+ if (VFLAG > 0) printf("test: estimated total sieving time = %s (with %d threads)\n\n",
time_from_secs(time, (unsigned long)score[i]), THREADS);
logprint(flog, "test: estimated total sieving time = %s (with %d threads)\n",
time_from_secs(time, (unsigned long)score[i]), THREADS);
@@ -400,8 +403,11 @@

fclose(flog);
remove(tmpbuf); // clean up after ourselves
- sprintf(tmpbuf, "%s", filenames[i]);
- remove(tmpbuf);
+ if (!are_files)
+ {
+ sprintf(tmpbuf, "%s", filenames[i]);
+ remove(tmpbuf);
+ }
sprintf(tmpbuf, "%s.afb.0", filenames[i]);
remove(tmpbuf);
}
@@ -409,6 +415,7 @@
// clean up memory allocated
if( are_files )
{
+ // TODO: need to write parameter adjustments to file
for(i = 0; i < njobs; i++)
{
mpz_polys_free(jobs[i].poly);

Index: top/driver.c
===================================================================
--- top/driver.c (revision 345)
+++ top/driver.c (working copy)
@@ -1133,7 +1133,7 @@
{
int ch = 0, i,j,valid;
char optbuf[MAXOPTIONLEN];
- char argbuf[80];
+ char argbuf[GSTR_MAXSIZE];

expression[0] = '\0';[/code]


If I do any further development of yafu, it will be by porting it to git and probably onward to GitHub (though I could be persuaded that there are better alternatives to GitHub) and largely ignoring svn/SourceForge. (If SourceForge hadn't gone to the dumps, I might have been willing to just use git's interface to svn, but all projects on SF need a new home anyways, so I don't plan to put up with svn either.)

Segfault with v1.34.5
 

My memory is weak. Is this something we visited before?

I am having a very repeatable segmentation fault. Here are two separate machines with different architecture, but the same trouble:

[code]
08/27/16 18:55:59 v1.34.5 @ math58, System/Build Info:
Using GMP-ECM 7.0.3, Powered by GMP 6.1.1
detected Intel(R) Core(TM)2 Duo CPU E7600 @ 3.06GHz
detected L1 = 32768 bytes, L2 = 3145728 bytes, CL = 64 bytes
measured cpu frequency ~= 3058.958720
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(1625417850674662961965677189663085535286500253815472)

fac: factoring 1625417850674662961965677189663085535286500253815472
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C41
rho: x^2 + 3, starting 1000 iterations on C36
rho: x^2 + 2, starting 1000 iterations on C36
rho: x^2 + 1, starting 1000 iterations on C36

starting SIQS on c36: 753954122664686097409379756097871777

==== sieving in progress ( 2 threads): 464 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)om 899 partial, (17235.29 rels/sec)
[/code][code]
08/27/16 20:42:08 v1.34.5 @ localhost.localdomain, System/Build Info:
Using GMP-ECM 7.0.3, Powered by GMP 6.1.1
detected AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
detected L1 = 65536 bytes, L2 = 1048576 bytes, CL = 64 bytes
measured cpu frequency ~= 1148.451510
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(1625417850674662961965677189663085535286500253815472)

fac: factoring 1625417850674662961965677189663085535286500253815472
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits

starting SIQS on c36: 753954122664686097409379756097871777

==== sieving in progress ( 2 threads): 464 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)rom 1615 partial, (15067.88 rels/sec)
[/code]More questions:

- Both of these machines have tune info in the respective yafu.ini files, yet both note, "fac: no tune info: using qs/gnfs crossover of 95 digits." Odd?

- Both are using 2 threads, which must have been chosen based on the yafu.ini files, yes?

- If these machines are running the same YAFU, based on the same ECM, based on the same GMP, why does the Intel machine give div and rho lines? Does it take a different route?

1.34.1 (which I still use) factors this particular composite without any issue.
Your issue may be ecm-related, as I don't see any ecm call, and calling siqs for such a small number may be out of siqs' parameters. Is your ecm path right?

[CODE]08/28/16 11:49:03 v1.34.1 @ XTREME-I7K, System/Build Info:
Using GMP-ECM, Powered by GMP
detected Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
measured cpu frequency ~= 3387.560070
using 20 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(1625417850674662961965677189663085535286500253815472)

fac: factoring 1625417850674662961965677189663085535286500253815472
fac: using pretesting plan: custom
fac: custom pretest ratio is: 0.3077
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C41
rho: x^2 + 2, starting 1000 iterations on C41
rho: x^2 + 2, starting 1000 iterations on C36
rho: x^2 + 1, starting 1000 iterations on C36
pm1: starting B1 = 150K, B2 = gmp-ecm default on C36
ecm: 5/30 curves on C36, B1=2K, B2=gmp-ecm default
Total factoring time = 0.0880 seconds


***factors found***

P1 = 2
P1 = 2
P1 = 2
P1 = 2
P1 = 3
P1 = 3
P3 = 379
P3 = 461
P4 = 1153
P5 = 74317
P12 = 269899936147
P25 = 2793457951223996505799291

ans = 1

>>[/CODE]