Can stages of polynomial selection be run separately

jux · 2016-01-13, 07:07

That is, is it possible to run -np1, -nps, and -npr from YAFU? If not, it would also be nice to see YAFU that could use GPU msieve. I run -nps with 8 threads in msieve but I can't tell if it's actually using all 8 because CPU usage is near 0%.

VBCurtis · 2016-01-13, 16:37

-nps isn't threaded in msieve; do you mean you run 8 instances of msieve? Most of the time, running -nps and -np1 at the same time for GPU-enabled msieve is most efficient, as -nps can easily keep up with the GPU (most settings- it's possible to have the GPU generate so many hits that nps can't keep up on a slow CPU, for some sizes of numbers).

YAFU is meant to automate smaller NFS runs; once you get up around 150 digits, the benefits (time savings) of hand-tuning the sieve parameters and poly-select settings outweigh the convenience of YAFU. If you're not over 140 digits, the GPU isn't all that big a deal- one user here claims CPU > GPU for even projects in the 140s.

jux · 2016-01-13, 20:29

I am currently factoring a 155 digit number as a benchmark. I ran -np1, -nps, and -npr all separately. I gave nearly 20 hours for -np1; I believe I lost more time from using msieve GPU and letting it run polynomial selection for too long rather than just using YAFU. Oh well. Your post was helpful either way, and I always learn something new from reading this forum.

VBCurtis · 2016-01-13, 21:14

tl;dr version: I think you did just what you should have for efficiency, and you learned a bit too.

A C155 should take about 40 thread-days to factor; using the traditional (among those who know more than I) rule of thumb in the forum of minimum 3% of expected factorization time on poly select, that's 1.2 thread-days at minimum for the 3 phases of poly-select. I think your 20 hrs of GPU time about right, though I personally would have given 36-48 hrs on the GPU (since my GPU doesn't do a whole lot, while I always have a queue of CPU tasks, I'm willing to "waste" time on GPU to save some CPU time).

I don't know how long -nps on its own would have taken, since I invoke msieve with -np1 -nps, but I would then have sorted (sort -g -k 10) the .ms file and run -npr on the top 100 or 200 lines (2ish hrs).

The 3% rule of thumb is open to interpretation with GPU time- a GPU does much more per unit time than a CPU, but we're just not sure how to account for that. At least, I haven't heard the professionals claim "with GPU on task size N, do y days of GPU".

jux · 2016-01-13, 22:09

I have run -npr for about 2 hours.
I'm now on the step where I convert msieve.fb into nfs.job. I guess I have to manually change N to n:, SKEW to skew:, R0 and R1 to Y0: and Y1:, and A0 through A5 to c0: through c5:?

VBCurtis · 2016-01-13, 23:16

When I run the python script factmsieve.py, the script converts the details automagically. I haven't run YAFU in many years; I can't say for sure what you need to convert for YAFU to pick up your poly-select work. I wouldn't be surprised if you can just fire it up; YAFU is pretty smart, and at worst you'll just get an error telling you what isn't found.

RichD · 2016-01-13, 23:22

Next time you can add the "-v" (or "-v -v") switch so you have a visual on the progress.

jux · 2016-01-14, 10:36

Quote:

Originally Posted by VBCurtis

When I run the python script factmsieve.py, the script converts the details automagically. I haven't run YAFU in many years; I can't say for sure what you need to convert for YAFU to pick up your poly-select work. I wouldn't be surprised if you can just fire it up; YAFU is pretty smart, and at worst you'll just get an error telling you what isn't found.

I suggest you give it a go again. I've found YAFU 1.34 to be faster than when I last used factmsieve.py (last year). The renaming scheme appears to have worked so I will stick with that.
Unrelated: I don't plan on doing another large factorization for a while. I don't think running my CPU at 100% for hundreds of hours is good for the computer.

fivemack · 2016-01-14, 10:42

I have run five computers doing factorization at 100% CPU for tens of thousands of hours now, with no detectable damage to any of the computers.

The electricity costs money, but desktop computers are still designed to run flat-out 24/365.

Dubslow · 2016-01-14, 11:01

If they do not, then some part of the computer is defective (or perhaps needs cleaning from dust).

jux · 2016-01-14, 11:05

Quote:

Originally Posted by fivemack

I have run five computers doing factorization at 100% CPU for tens of thousands of hours now, with no detectable damage to any of the computers.

The electricity costs money, but desktop computers are still designed to run flat-out 24/365.

That's good to hear.

2016-01-13, 07:07	#1
jux Aug 2015 110110₂ Posts	Can stages of polynomial selection be run separately That is, is it possible to run -np1, -nps, and -npr from YAFU? If not, it would also be nice to see YAFU that could use GPU msieve. I run -nps with 8 threads in msieve but I can't tell if it's actually using all 8 because CPU usage is near 0%. Last fiddled with by jux on 2016-01-13 at 07:13

2016-01-13, 20:29	#3
jux Aug 2015 2·3³ Posts	I am currently factoring a 155 digit number as a benchmark. I ran -np1, -nps, and -npr all separately. I gave nearly 20 hours for -np1; I believe I lost more time from using msieve GPU and letting it run polynomial selection for too long rather than just using YAFU. Oh well. Your post was helpful either way, and I always learn something new from reading this forum. Last fiddled with by jux on 2016-01-13 at 20:30

2016-01-13, 22:09	#5
jux Aug 2015 110110₂ Posts	I have run -npr for about 2 hours. I'm now on the step where I convert msieve.fb into nfs.job. I guess I have to manually change N to n:, SKEW to skew:, R0 and R1 to Y0: and Y1:, and A0 through A5 to c0: through c5:? Last fiddled with by jux on 2016-01-13 at 22:09

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Polynomial selection	Max0526	NFS@Home	9	2017-05-20 08:57
Updated polynomial selection	jasonp	Msieve	65	2011-05-01 19:06
GNFS polynomial selection	Unregistered	Information & Answers	3	2011-04-16 14:24
2^877-1 polynomial selection	fivemack	Factoring	47	2009-06-16 00:24
Polynomial selection	CRGreathouse	Factoring	2	2009-05-25 07:55

2016-01-13, 16:37	#2
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	-nps isn't threaded in msieve; do you mean you run 8 instances of msieve? Most of the time, running -nps and -np1 at the same time for GPU-enabled msieve is most efficient, as -nps can easily keep up with the GPU (most settings- it's possible to have the GPU generate so many hits that nps can't keep up on a slow CPU, for some sizes of numbers). YAFU is meant to automate smaller NFS runs; once you get up around 150 digits, the benefits (time savings) of hand-tuning the sieve parameters and poly-select settings outweigh the convenience of YAFU. If you're not over 140 digits, the GPU isn't all that big a deal- one user here claims CPU > GPU for even projects in the 140s.

2016-01-13, 21:14	#4
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	tl;dr version: I think you did just what you should have for efficiency, and you learned a bit too. A C155 should take about 40 thread-days to factor; using the traditional (among those who know more than I) rule of thumb in the forum of minimum 3% of expected factorization time on poly select, that's 1.2 thread-days at minimum for the 3 phases of poly-select. I think your 20 hrs of GPU time about right, though I personally would have given 36-48 hrs on the GPU (since my GPU doesn't do a whole lot, while I always have a queue of CPU tasks, I'm willing to "waste" time on GPU to save some CPU time). I don't know how long -nps on its own would have taken, since I invoke msieve with -np1 -nps, but I would then have sorted (sort -g -k 10) the .ms file and run -npr on the top 100 or 200 lines (2ish hrs). The 3% rule of thumb is open to interpretation with GPU time- a GPU does much more per unit time than a CPU, but we're just not sure how to account for that. At least, I haven't heard the professionals claim "with GPU on task size N, do y days of GPU".

2016-01-13, 23:16	#6
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	When I run the python script factmsieve.py, the script converts the details automagically. I haven't run YAFU in many years; I can't say for sure what you need to convert for YAFU to pick up your poly-select work. I wouldn't be surprised if you can just fire it up; YAFU is pretty smart, and at worst you'll just get an error telling you what isn't found.

2016-01-13, 23:22	#7
RichD Sep 2008 Kansas 2⁴·211 Posts	Next time you can add the "-v" (or "-v -v") switch so you have a visual on the progress.

2016-01-14, 10:42	#9
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 14423₈ Posts	I have run five computers doing factorization at 100% CPU for tens of thousands of hours now, with no detectable damage to any of the computers. The electricity costs money, but desktop computers are still designed to run flat-out 24/365.

2016-01-14, 11:01	#10
Dubslow Basketry That Evening! "Bunslow the Bold" Jun 2011 40<A<43 -89<O<-88 3×29×83 Posts	If they do not, then some part of the computer is defective (or perhaps needs cleaning from dust).