Newb question - Page 2

PicGrabber · 2014-10-29, 15:43

Hey Guys,

Thank you all for help !!!!!

After all, pointing to me, the process was wasting time doing same thing 4 times, was enough to understand something was wrong... I was sure 2 months for a 512 RSA was taking too long, I wasn't able to know why, by myself.

Well, assuming starting all over again should take something like 2 weeks on a 16 core cpu, I have decided to start all over again

I promise I'll be back here in ~2 weeks to say thanks again when it get done.

OR...... if it get stuck again

Really thanks for your help and attention !!!

PicGrabber · 2014-10-29, 18:54

Quote:

Originally Posted by Batalov

One thing that the script is equipped to do is restart. You can kill the existing script.
Then backup the whole directory, just in case something goes wrong later. You will "bite your elbows", as the Russian saying goes, if you don't.

Then you have to examine the script and if you know a bit of perl (python), change it so that it does indeed prepare 1/16th slices of each chunk of work. Or you can try to get the latest script from http://sourceforge.net/p/ggnfs/code/.../factMsieve.pl and be sure to edit to set CPU number to 16 and then start the new copy. You can post the similar log output later and we will check if this worked.

By the way Batalov, I have restarted using the perl - factMsieve.pl you pointed, instead of the phyton - factMsieve.py I was using before......
Have updated the file with the correct path and changed it to 16 processors.

Don't know if it's a better approach, but let's try

Thanks.

Batalov · 2014-10-29, 20:34

It should be better.

You wouldn't really want to abandon a ~50% done project. These are fine relations; they are just redundant, not broken, not junk.

PicGrabber · 2014-10-30, 12:16

Quote:

Originally Posted by Batalov

It should be better.

You wouldn't really want to abandon a ~50% done project. These are fine relations; they are just redundant, not broken, not junk.

OK, I have run the remdups tool, at the backup I have done, dat file went down from 15Gb to 3Gb, lots of dup.

I have resumed the process, to check how it goes...

So I'm running a new process from zero and in parallel the old one with a scrubbed dat file. I'll post the log file soon.

Thanks

PicGrabber · 2014-10-30, 14:36

Well, seems to be repeating all over again.

Thu Oct 30 12:00:10 2014 -> factmsieve.py (v0.76)
Thu Oct 30 12:00:10 2014 -> This is client 1 of 1
Thu Oct 30 12:00:10 2014 -> Running on 16 Cores with 1 hyper-thread per Core
Thu Oct 30 12:00:10 2014 -> Working with NAME = Factr
Thu Oct 30 12:00:10 2014 -> Selected lattice siever: gnfs-lasieve4I14e
Thu Oct 30 12:00:10 2014 -> No parameter change detected, resuming...
Thu Oct 30 12:00:10 2014 -> Running setup ...
Thu Oct 30 12:00:10 2014 -> Estimated minimum relations needed: 5.70292e+07
Thu Oct 30 12:00:10 2014 -> resuming a block for q from 161000000 to 161100000
Thu Oct 30 12:00:10 2014 -> Running lattice siever ...
Thu Oct 30 12:00:10 2014 -> entering sieving loop
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004863 in 161000000 .. 161012328 as file Factr.job.T0
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004869 in 161000000 .. 161012385 as file Factr.job.T1
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004763 in 161000000 .. 161012442 as file Factr.job.T2
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004847 in 161000000 .. 161012500 as file Factr.job.T3
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161024915 as file Factr.job.T4
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161024943 as file Factr.job.T5
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017243 in 161012500 .. 161024971 as file Factr.job.T6
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161025000 as file Factr.job.T7
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037491 as file Factr.job.T8
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037494 as file Factr.job.T9
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037497 as file Factr.job.T10
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037500 as file Factr.job.T11
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042677 in 161037500 .. 161049948 as file Factr.job.T12
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042681 in 161037500 .. 161049965 as file Factr.job.T13
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042681 in 161037500 .. 161049982 as file Factr.job.T14
Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042677 in 161037500 .. 161050000 as file Factr.job.T15
Thu Oct 30 12:00:10 2014 -> Lattice sieving algebraic q from 161000000 to 161100000.
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14
Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15
Thu Oct 30 12:38:57 2014 Found 28681951 relations, 50.3% of the estimated minimum (57029236).
Thu Oct 30 12:38:57 2014 LatSieveTime: 2326.9
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112328 as file Factr.job.T0
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112385 as file Factr.job.T1
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112442 as file Factr.job.T2
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112500 as file Factr.job.T3
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124915 as file Factr.job.T4
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124943 as file Factr.job.T5
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124971 as file Factr.job.T6
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161125000 as file Factr.job.T7
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137491 as file Factr.job.T8
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137494 as file Factr.job.T9
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137497 as file Factr.job.T10
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137500 as file Factr.job.T11
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149948 as file Factr.job.T12
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149965 as file Factr.job.T13
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149982 as file Factr.job.T14
Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161150000 as file Factr.job.T15
Thu Oct 30 12:38:57 2014 -> Lattice sieving algebraic q from 161100000 to 161200000.
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14
Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15
Thu Oct 30 13:43:53 2014 Found 28745033 relations, 50.4% of the estimated minimum (57029236).
Thu Oct 30 13:43:53 2014 LatSieveTime: 3896.11
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212328 as file Factr.job.T0
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212385 as file Factr.job.T1
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212442 as file Factr.job.T2
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212500 as file Factr.job.T3
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224915 as file Factr.job.T4
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224943 as file Factr.job.T5
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224971 as file Factr.job.T6
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161225000 as file Factr.job.T7
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237491 as file Factr.job.T8
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237494 as file Factr.job.T9
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237497 as file Factr.job.T10
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237500 as file Factr.job.T11
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249948 as file Factr.job.T12
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249965 as file Factr.job.T13
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249982 as file Factr.job.T14
Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161250000 as file Factr.job.T15

VBCurtis · 2014-10-30, 16:29

The data you posted starts with "resuming". If you try the factorization again with any file other than the polynomial in the folder, the resume/checkpoint notes may still be there, which include the confused range-splitting setup that caused you the problem in the first place. This is why I suggested starting over fresh, rather than trying to salvage the previous run- the intricacies of setting Q-values, splitting ranges, making sure the script doesn't re-sieve a region you already have relations for are challenging to a new user of the script. A fresh start is "supposed" to just work, even though it will repeat finding the 3GB of relations you already have.

I don't think Q-values of 160M would be where the script begins for a number of this size, which also suggests your new run picked up the status/resume data from your old run. I'd leave this folder alone, make a new folder, put the poly file in there, and start anew.

Batalov · 2014-10-30, 16:58

Quote:

Originally Posted by PicGrabber

Well, seems to be repeating all over again.

Thu Oct 30 12:00:10 2014 -> factmsieve.py (v0.76)
...

There is a large thread about the python script. Check if there were any solutions to your problem posted there.

I've never used this python script; I have long ago used the perl script and then switched to my own script (with qsub()s etc)

PicGrabber · 2014-10-31, 11:45

Quote:

Originally Posted by VBCurtis

The data you posted starts with "resuming". If you try the factorization again with any file other than the polynomial in the folder, the resume/checkpoint notes may still be there, which include the confused range-splitting setup that caused you the problem in the first place. This is why I suggested starting over fresh, rather than trying to salvage the previous run- the intricacies of setting Q-values, splitting ranges, making sure the script doesn't re-sieve a region you already have relations for are challenging to a new user of the script. A fresh start is "supposed" to just work, even though it will repeat finding the 3GB of relations you already have.

I don't think Q-values of 160M would be where the script begins for a number of this size, which also suggests your new run picked up the status/resume data from your old run. I'd leave this folder alone, make a new folder, put the poly file in there, and start anew.

There you go VBCurtis, no repetition.

Code:

Thu Oct 30 18:16:43 2014 -> factmsieve.py (v0.76)
Thu Oct 30 18:16:43 2014 -> This is client 1 of 1
Thu Oct 30 18:16:43 2014 -> Running on 16 Cores with 1 hyper-thread per Core
Thu Oct 30 18:16:43 2014 -> Working with NAME = Factr
Thu Oct 30 18:16:43 2014 -> Selected lattice siever: gnfs-lasieve4I14e
Thu Oct 30 18:16:43 2014 -> Creating param file to detect parameter changes...
Thu Oct 30 18:16:43 2014 -> Running setup ...
Thu Oct 30 18:16:43 2014 -> Estimated minimum relations needed: 5.70292e+07
Thu Oct 30 18:16:43 2014 -> cleaning up before a restart
Thu Oct 30 18:16:43 2014 -> Running lattice siever ...
Thu Oct 30 18:16:43 2014 -> entering sieving loop
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12850000 in 12850000 .. 12856250 as file Factr.job.T0
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12856250 in 12856250 .. 12862500 as file Factr.job.T1
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12862500 in 12862500 .. 12868750 as file Factr.job.T2
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12868750 in 12868750 .. 12875000 as file Factr.job.T3
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12875000 in 12875000 .. 12881250 as file Factr.job.T4
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12881250 in 12881250 .. 12887500 as file Factr.job.T5
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12887500 in 12887500 .. 12893750 as file Factr.job.T6
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12893750 in 12893750 .. 12900000 as file Factr.job.T7
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12900000 in 12900000 .. 12906250 as file Factr.job.T8
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12906250 in 12906250 .. 12912500 as file Factr.job.T9
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12912500 in 12912500 .. 12918750 as file Factr.job.T10
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12918750 in 12918750 .. 12925000 as file Factr.job.T11
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12925000 in 12925000 .. 12931250 as file Factr.job.T12
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12931250 in 12931250 .. 12937500 as file Factr.job.T13
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12937500 in 12937500 .. 12943750 as file Factr.job.T14
Thu Oct 30 18:16:43 2014 -> making sieve job for q = 12943750 in 12943750 .. 12950000 as file Factr.job.T15
Thu Oct 30 18:16:43 2014 -> Lattice sieving algebraic q from 12850000 to 12950000.
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14
Thu Oct 30 18:16:43 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15
Thu Oct 30 18:48:59 2014 Found 17333 relations, 0.0% of the estimated minimum (57029236).
Thu Oct 30 18:48:59 2014 LatSieveTime: 1936.46
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12950000 in 12950000 .. 12956250 as file Factr.job.T0
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12956250 in 12956250 .. 12962500 as file Factr.job.T1
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12962500 in 12962500 .. 12968750 as file Factr.job.T2
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12968750 in 12968750 .. 12975000 as file Factr.job.T3
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12975000 in 12975000 .. 12981250 as file Factr.job.T4
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12981250 in 12981250 .. 12987500 as file Factr.job.T5
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12987500 in 12987500 .. 12993750 as file Factr.job.T6
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 12993750 in 12993750 .. 13000000 as file Factr.job.T7
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13000000 in 13000000 .. 13006250 as file Factr.job.T8
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13006250 in 13006250 .. 13012500 as file Factr.job.T9
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13012500 in 13012500 .. 13018750 as file Factr.job.T10
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13018750 in 13018750 .. 13025000 as file Factr.job.T11
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13025000 in 13025000 .. 13031250 as file Factr.job.T12
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13031250 in 13031250 .. 13037500 as file Factr.job.T13
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13037500 in 13037500 .. 13043750 as file Factr.job.T14
Thu Oct 30 18:48:59 2014 -> making sieve job for q = 13043750 in 13043750 .. 13050000 as file Factr.job.T15
Thu Oct 30 18:48:59 2014 -> Lattice sieving algebraic q from 12950000 to 13050000.
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14
Thu Oct 30 18:48:59 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15
Thu Oct 30 19:21:37 2014 Found 35053 relations, 0.1% of the estimated minimum (57029236).

RichD · 2014-10-31, 14:39

17K-rels/100K-Q seems like an awfully low yield.
Are you sure you are combining all 16 jobs?

VBCurtis · 2014-10-31, 20:06

Rich is right; something is still not so good here. One possibility is that your polynomial is not 'good' for this job; A rule of thumb (though perhaps not optimal- we don't really know) is to spend 3-5% of the overall CPU time expected for a job on the poly search. A 1% improvement in Murphy E-value (aka Score) may mean 1% less time sieving (it's not quite linear, but roughly so for small improvements in score).

If you have a look at the Polynomial Request Thread, you can find a few GNFS requests in the C154 or C155 range, and see what scores are typical from a GPU poly search. If your poly is not within a factor of two of those scores, you should abandon the job with this poly, put a day into finding a new one (or request one in that thread- I can find one overnight tonight for you), and start fresh again.

Typical 'yield' is from 1.5 to 4 relations per Q; so those intervals of 100k from the default script should yield you 150k to 400k relations. You got 17k and 18k, VERY low. Basically, the job should be going 10 times faster!

Sometimes, a terribly low yield happens when parameters are chosen poorly by an individual, or by the script. But, your log posts show it selected the 14e siever, which is correct for this job; and 57M relations needed is typical for a 30-bit large primes job, also reasonable for a job this size. Those two choices, with a decent poly, should produce yield around 2.0 initially, falling to 1.5 or a bit lower toward the end of the job. Your yield is 0.175 so far, a long ways from acceptable- so long that the job might not finish.

Notice the script on a fresh run started around Q=12M; If yield is around 2.0, 80M relations means you would sieve from 12M to 52M, roughly. That's 400 of the 100k intervals; from those guesses, you can estimate how long the sieve phase will take. Of course, yield might be 1.6 or 2.5, so my guess might be off by 25% or more!
Edit: Just saw 1900 sec for the first 100k batch. 400 of those batches would take you around 200 hours for the sieve phase, once the yield problem is fixed.
Edit2: If Rich is right and the 16 threads are not combining relations into a single pool, you'll see that each of your threads is finding 15000 or 16000 relations as you close in on the end of the interval; but after saving and processing, the 16 batches of 16k relations aren't going to one big bucket of ~250k relations (which, note, would be 2.5 yield, possible). So, you might have a good poly, but something is wrong with the script such that the 16 pools of relations aren't being all saved into one place after each 100k interval completes.
-Curtis

2014-10-31, 20:06	#21
VBCurtis "Curtis" Feb 2005 Riverside, CA 2·2,927 Posts	Rich is right; something is still not so good here. One possibility is that your polynomial is not 'good' for this job; A rule of thumb (though perhaps not optimal- we don't really know) is to spend 3-5% of the overall CPU time expected for a job on the poly search. A 1% improvement in Murphy E-value (aka Score) may mean 1% less time sieving (it's not quite linear, but roughly so for small improvements in score). If you have a look at the Polynomial Request Thread, you can find a few GNFS requests in the C154 or C155 range, and see what scores are typical from a GPU poly search. If your poly is not within a factor of two of those scores, you should abandon the job with this poly, put a day into finding a new one (or request one in that thread- I can find one overnight tonight for you), and start fresh again. Typical 'yield' is from 1.5 to 4 relations per Q; so those intervals of 100k from the default script should yield you 150k to 400k relations. You got 17k and 18k, VERY low. Basically, the job should be going 10 times faster! Sometimes, a terribly low yield happens when parameters are chosen poorly by an individual, or by the script. But, your log posts show it selected the 14e siever, which is correct for this job; and 57M relations needed is typical for a 30-bit large primes job, also reasonable for a job this size. Those two choices, with a decent poly, should produce yield around 2.0 initially, falling to 1.5 or a bit lower toward the end of the job. Your yield is 0.175 so far, a long ways from acceptable- so long that the job might not finish. Notice the script on a fresh run started around Q=12M; If yield is around 2.0, 80M relations means you would sieve from 12M to 52M, roughly. That's 400 of the 100k intervals; from those guesses, you can estimate how long the sieve phase will take. Of course, yield might be 1.6 or 2.5, so my guess might be off by 25% or more! Edit: Just saw 1900 sec for the first 100k batch. 400 of those batches would take you around 200 hours for the sieve phase, once the yield problem is fixed. Edit2: If Rich is right and the 16 threads are not combining relations into a single pool, you'll see that each of your threads is finding 15000 or 16000 relations as you close in on the end of the interval; but after saving and processing, the 16 batches of 16k relations aren't going to one big bucket of ~250k relations (which, note, would be 2.5 yield, possible). So, you might have a good poly, but something is wrong with the script such that the 16 pools of relations aren't being all saved into one place after each 100k interval completes. -Curtis Last fiddled with by VBCurtis on 2014-10-31 at 20:14

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Newb who needs help with PC	EddieTheBear	Hardware	19	2015-10-23 13:22
100MD LL Test - Newb Question	Neo	Information & Answers	6	2011-01-02 16:01
Questions from a GIMPS newb	hj47	Hardware	10	2008-10-28 20:00
Newb help (it crashes)	Proggie	Software	4	2005-01-05 07:35
linux question ( newb)	crash893	Software	2	2003-12-26 18:50

2014-10-29, 15:43	#12
PicGrabber Oct 2014 7 Posts	Hey Guys, Thank you all for help !!!!! After all, pointing to me, the process was wasting time doing same thing 4 times, was enough to understand something was wrong... I was sure 2 months for a 512 RSA was taking too long, I wasn't able to know why, by myself. Well, assuming starting all over again should take something like 2 weeks on a 16 core cpu, I have decided to start all over again I promise I'll be back here in ~2 weeks to say thanks again when it get done. OR...... if it get stuck again Really thanks for your help and attention !!!

2014-10-29, 20:34	#14
Batalov "Serge" Mar 2008 San Diego, Calif. 3²×7×163 Posts	It should be better. You wouldn't really want to abandon a ~50% done project. These are fine relations; they are just redundant, not broken, not junk.

2014-10-30, 14:36	#16
PicGrabber Oct 2014 111₂ Posts	Well, seems to be repeating all over again. Thu Oct 30 12:00:10 2014 -> factmsieve.py (v0.76) Thu Oct 30 12:00:10 2014 -> This is client 1 of 1 Thu Oct 30 12:00:10 2014 -> Running on 16 Cores with 1 hyper-thread per Core Thu Oct 30 12:00:10 2014 -> Working with NAME = Factr Thu Oct 30 12:00:10 2014 -> Selected lattice siever: gnfs-lasieve4I14e Thu Oct 30 12:00:10 2014 -> No parameter change detected, resuming... Thu Oct 30 12:00:10 2014 -> Running setup ... Thu Oct 30 12:00:10 2014 -> Estimated minimum relations needed: 5.70292e+07 Thu Oct 30 12:00:10 2014 -> resuming a block for q from 161000000 to 161100000 Thu Oct 30 12:00:10 2014 -> Running lattice siever ... Thu Oct 30 12:00:10 2014 -> entering sieving loop Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004863 in 161000000 .. 161012328 as file Factr.job.T0 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004869 in 161000000 .. 161012385 as file Factr.job.T1 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004763 in 161000000 .. 161012442 as file Factr.job.T2 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161004847 in 161000000 .. 161012500 as file Factr.job.T3 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161024915 as file Factr.job.T4 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161024943 as file Factr.job.T5 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017243 in 161012500 .. 161024971 as file Factr.job.T6 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161017201 in 161012500 .. 161025000 as file Factr.job.T7 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037491 as file Factr.job.T8 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037494 as file Factr.job.T9 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037497 as file Factr.job.T10 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161029639 in 161025000 .. 161037500 as file Factr.job.T11 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042677 in 161037500 .. 161049948 as file Factr.job.T12 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042681 in 161037500 .. 161049965 as file Factr.job.T13 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042681 in 161037500 .. 161049982 as file Factr.job.T14 Thu Oct 30 12:00:10 2014 -> making sieve job for q = 161042677 in 161037500 .. 161050000 as file Factr.job.T15 Thu Oct 30 12:00:10 2014 -> Lattice sieving algebraic q from 161000000 to 161100000. Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14 Thu Oct 30 12:00:10 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15 Thu Oct 30 12:38:57 2014 Found 28681951 relations, 50.3% of the estimated minimum (57029236). Thu Oct 30 12:38:57 2014 LatSieveTime: 2326.9 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112328 as file Factr.job.T0 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112385 as file Factr.job.T1 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112442 as file Factr.job.T2 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161100000 in 161100000 .. 161112500 as file Factr.job.T3 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124915 as file Factr.job.T4 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124943 as file Factr.job.T5 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161124971 as file Factr.job.T6 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161112500 in 161112500 .. 161125000 as file Factr.job.T7 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137491 as file Factr.job.T8 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137494 as file Factr.job.T9 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137497 as file Factr.job.T10 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161125000 in 161125000 .. 161137500 as file Factr.job.T11 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149948 as file Factr.job.T12 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149965 as file Factr.job.T13 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161149982 as file Factr.job.T14 Thu Oct 30 12:38:57 2014 -> making sieve job for q = 161137500 in 161137500 .. 161150000 as file Factr.job.T15 Thu Oct 30 12:38:57 2014 -> Lattice sieving algebraic q from 161100000 to 161200000. Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T0 -v -n0 -a Factr.job.T0 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T1 -v -n1 -a Factr.job.T1 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T2 -v -n2 -a Factr.job.T2 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T3 -v -n3 -a Factr.job.T3 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T4 -v -n4 -a Factr.job.T4 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T5 -v -n5 -a Factr.job.T5 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T6 -v -n6 -a Factr.job.T6 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T7 -v -n7 -a Factr.job.T7 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T8 -v -n8 -a Factr.job.T8 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T9 -v -n9 -a Factr.job.T9 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T10 -v -n10 -a Factr.job.T10 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T11 -v -n11 -a Factr.job.T11 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T12 -v -n12 -a Factr.job.T12 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T13 -v -n13 -a Factr.job.T13 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T14 -v -n14 -a Factr.job.T14 Thu Oct 30 12:38:57 2014 -> gnfs-lasieve4I14e -k -o spairs.out.T15 -v -n15 -a Factr.job.T15 Thu Oct 30 13:43:53 2014 Found 28745033 relations, 50.4% of the estimated minimum (57029236). Thu Oct 30 13:43:53 2014 LatSieveTime: 3896.11 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212328 as file Factr.job.T0 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212385 as file Factr.job.T1 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212442 as file Factr.job.T2 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161200000 in 161200000 .. 161212500 as file Factr.job.T3 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224915 as file Factr.job.T4 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224943 as file Factr.job.T5 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161224971 as file Factr.job.T6 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161212500 in 161212500 .. 161225000 as file Factr.job.T7 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237491 as file Factr.job.T8 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237494 as file Factr.job.T9 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237497 as file Factr.job.T10 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161225000 in 161225000 .. 161237500 as file Factr.job.T11 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249948 as file Factr.job.T12 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249965 as file Factr.job.T13 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161249982 as file Factr.job.T14 Thu Oct 30 13:43:53 2014 -> making sieve job for q = 161237500 in 161237500 .. 161250000 as file Factr.job.T15

2014-10-30, 16:29	#17
VBCurtis "Curtis" Feb 2005 Riverside, CA 2×2,927 Posts	The data you posted starts with "resuming". If you try the factorization again with any file other than the polynomial in the folder, the resume/checkpoint notes may still be there, which include the confused range-splitting setup that caused you the problem in the first place. This is why I suggested starting over fresh, rather than trying to salvage the previous run- the intricacies of setting Q-values, splitting ranges, making sure the script doesn't re-sieve a region you already have relations for are challenging to a new user of the script. A fresh start is "supposed" to just work, even though it will repeat finding the 3GB of relations you already have. I don't think Q-values of 160M would be where the script begins for a number of this size, which also suggests your new run picked up the status/resume data from your old run. I'd leave this folder alone, make a new folder, put the poly file in there, and start anew.

2014-10-31, 14:39	#20
RichD Sep 2008 Kansas 111101110001₂ Posts	17K-rels/100K-Q seems like an awfully low yield. Are you sure you are combining all 16 jobs?