CADO NFS - Page 18

EdH · 2018-05-04, 03:18

Curtis,

I've added another machine to my mix and hope to rework some of my scripts in the next few days. I'm trying to see if there is a way to fit a 160 into my daytime on time for several of the machines. I think you came up with a doubling of effort at something over 5 digits with cado-nfs. I calculated the poly select and sieving at just over 7 hours for the C150. This only allows for one doubling, if that, within my limited window.

How do you think your C150 params would run for a C155, if I try to see how it would compare to the C150, realizing an added machine would skew any direct comparison?

Ed

VBCurtis · 2018-05-04, 04:06

Ed-
I'll have a C155 best-guess file for you in a couple hours. I won't change much from the C150, but I'll try to make it a bit better than the existing C150 params would be.

If you happen to still have the log from the good C150 run, try to dig through the file to discover the last Q searched? That info gives me an idea of whether the alim/rlim choices need to be inflated a bit for a larger factorization. I'm hoping this final-Q is around 15-18M.
If I get the lim's set correctly for C155, you should manage to come in under double the time for C150. I realized that I can make poly select a bit more efficient for your mega-multi-process setup; I'll put that into the params file too. It's a simple change: poly select was previously split into segments of 500, with CADO searching every multiple of 60. If we change that to 480, each process will always search 8 coefficients, instead of some receiving 9 to split into two threads. Fewer single-threaded running will take place, so wall-clock time will improve a bit for the same CPU-time.

wombatman · 2018-05-04, 04:08

Curtis, would you be willing to go ahead and take a stab at a C165 parameter file (or suggest what you think I could change to improve on the default)? I have a C164 I want to tackle with CADO, but given that you were able to cut the time on the C150 in half, I think it would be more than worthwhile to try and do the same for the C164.

VBCurtis · 2018-05-04, 04:17

Sure! I think I got lucky on the C150, as it's easier to extrapolate on slightly-smaller jobs, but I'll be happy to give it a shot. Having you guys test my settings makes refining parameters a whole lot less of a grind for me.
I'll post both after I finish the Winnipeg/Nashville game on my DVR, likely by 11PM PDT.

VBCurtis · 2018-05-04, 05:41

Ed-
Here's my C155 draft params:

Code:

###########################################################################
# Polynomial selection
###########################################################################

tasks.polyselect.degree = 5

tasks.polyselect.P = 650000
tasks.polyselect.admin = 1020
tasks.polyselect.admax = 22e4
tasks.polyselect.adrange = 480
tasks.polyselect.incr = 60
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 100
tasks.polyselect.ropteffort = 13

###########################################################################
# Sieve
###########################################################################

lim0 = 18000000
lim1 = 33000000
lpb0 = 30
lpb1 = 31
tasks.sieve.mfb0 = 60
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 17
tasks.sieve.ncurves1 = 25
tasks.I = 14

tasks.sieve.qrange = 2000
tasks.sieve.qmin = 4500000

###########################################################################
# Filtering
###########################################################################

tasks.filter.purge.keep = 175
tasks.filter.maxlevel = 30
tasks.filter.target_density = 155.0

This should spend about the same time on poly select as the C150 did. Sieving will hopefully be just under twice as long, and no idea what the matrix will do.
If you get a chance on the console output to note the last Q sieved, I would appreciate that info in addition to the stats you previously reported. If yield is similar to an old G155 GGNFS run I found a log for, it'll be around 28M. I expect yield to be better than GGNFS, so hopefully your max-Q will be below 25M.
Good luck!

VBCurtis · 2018-05-04, 05:56

wombatman-
Here's my guess at c165 params:

Code:

###########################################################################
# Polynomial selection
###########################################################################

tasks.polyselect.degree = 5

tasks.polyselect.P = 1200000
tasks.polyselect.admin = 1500
tasks.polyselect.admax = 6e5
tasks.polyselect.adrange = 960
tasks.polyselect.incr = 60
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 100
tasks.polyselect.ropteffort = 15

###########################################################################
# Sieve
###########################################################################

tasks.I = 14
tasks.sieve.qmin = 7000000
lim0 = 38000000
lim1 = 60000000
lpb0 = 31
lpb1 = 32
tasks.sieve.mfb0 = 62
tasks.sieve.mfb1 = 64
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 2000

###########################################################################
# Filtering
###########################################################################

tasks.filter.purge.keep = 175
tasks.filter.maxlevel = 32
tasks.filter.target_density = 160.0

Note that bwc.interval is just a "how often we update the screen" setting; I like it at 500 so I get more frequent screen output.
The poly select should take ~1.4M cpu-seconds, which is 4x longer than Ed's C155 file. I did a CADO poly select for C166 a month or two ago, and spent 1.55M cpu-seconds; I adjusted params a bit lower for your C164.
If you could report the CPU and wall-clock times for size-opt, root-opt, sieve, LA as Ed did, that would help me out for future optimization. If you happen to see the maximum Q sieved, that helps refine alim/rlim selection.

EdH · 2018-05-04, 13:19

Thanks Curtis,

I still might be a "couple" days before I get the scripts where I want them and I lost the drive on one of my i7s. Hopefully, just a minor inconvenience.

I was sure I kept the temporary directory, but it isn't there. However, I do have the last part of the terminal output. The last assignment says:

Code:

Sending workunit c150_sieving_15994000-15996000 to client

The thread default for all the clients is two. If I am not planning on the machines doing anything but cado-nfs, would it be more efficient to customize the threads based on the available processors to possibly one client using all threads? I could easily do this within my scripts.

I might be over thinking this, but what about the difference between machine capability? Let's say a Core2 Quad 2.4GHz (4 proc) vs. an i7 3.4GHz with HT (8 proc). Would it be better to load up a single client on both machines, or maybe give the Core2 a single loaded client and the i7 four, two thread clients for a total of five more balanced instances?

How much value does bogomips have?

Ed

VBCurtis · 2018-05-04, 15:26

Excellent! Q=16M is in the optimistic side of the expected range.

The only reason to adjust from 2 threads per process that I know of is to save memory. I=15 uses something on the order of 2GB per process, but the thread pool shares that footprint; a 4GB 4-core machine would want to run 4 threads on a single client rather than two on two clients to save memory when I=15 (Say, GNFS-180+). I believe I=14 uses around 500MB per client, which should be no problem even with multiple clients running.

EdH · 2018-05-04, 15:42

Quote:

Originally Posted by VBCurtis

Excellent! Q=16M is in the optimistic side of the expected range.

The only reason to adjust from 2 threads per process that I know of is to save memory. I=15 uses something on the order of 2GB per process, but the thread pool shares that footprint; a 4GB 4-core machine would want to run 4 threads on a single client rather than two on two clients to save memory when I=15 (Say, GNFS-180+). I believe I=14 uses around 500MB per client, which should be no problem even with multiple clients running.

Thanks. Most of my i7s (actually, the others, too) are memory deficient at 4GB. The server has 16GB.

wombatman · 2018-05-04, 17:37

Quote:

Originally Posted by VBCurtis

wombatman-
Here's my guess at c165 params:

Code:

###########################################################################
# Polynomial selection
###########################################################################

tasks.polyselect.degree = 5

tasks.polyselect.P = 1200000
tasks.polyselect.admin = 1500
tasks.polyselect.admax = 6e5
tasks.polyselect.adrange = 960
tasks.polyselect.incr = 60
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 100
tasks.polyselect.ropteffort = 15

###########################################################################
# Sieve
###########################################################################

tasks.I = 14
tasks.sieve.qmin = 7000000
lim0 = 38000000
lim1 = 60000000
lpb0 = 31
lpb1 = 32
tasks.sieve.mfb0 = 62
tasks.sieve.mfb1 = 64
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 2000

###########################################################################
# Filtering
###########################################################################

tasks.filter.purge.keep = 175
tasks.filter.maxlevel = 32
tasks.filter.target_density = 160.0

Note that bwc.interval is just a "how often we update the screen" setting; I like it at 500 so I get more frequent screen output.
The poly select should take ~1.4M cpu-seconds, which is 4x longer than Ed's C155 file. I did a CADO poly select for C166 a month or two ago, and spent 1.55M cpu-seconds; I adjusted params a bit lower for your C164.
If you could report the CPU and wall-clock times for size-opt, root-opt, sieve, LA as Ed did, that would help me out for future optimization. If you happen to see the maximum Q sieved, that helps refine alim/rlim selection.

Thanks Curtis. I'll try and get this started tonight and report on all requested times when it finishes.

VictordeHolland · 2018-05-05, 12:43

Does anybody got CADO-NFS working under windoz?
On Ubuntu it is super easy with "git clone" and "make" and it will get all the packages it needs.

I compiled GMP, ECM and mlucas on Windows using Mingw, but this is a totally different beast. Even with the instructions included in the git repo :(. But I'm still a newbie with compiling....

2018-05-04, 04:17	#191
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	Sure! I think I got lucky on the C150, as it's easier to extrapolate on slightly-smaller jobs, but I'll be happy to give it a shot. Having you guys test my settings makes refining parameters a whole lot less of a grind for me. I'll post both after I finish the Winnipeg/Nashville game on my DVR, likely by 11PM PDT. Last fiddled with by VBCurtis on 2018-05-04 at 04:17

2018-05-04, 13:19	#194
EdH "Ed Hall" Dec 2009 Adirondack Mtns 3817₁₀ Posts	Thanks Curtis, I still might be a "couple" days before I get the scripts where I want them and I lost the drive on one of my i7s. Hopefully, just a minor inconvenience. I was sure I kept the temporary directory, but it isn't there. However, I do have the last part of the terminal output. The last assignment says: Code: Sending workunit c150_sieving_15994000-15996000 to client The thread default for all the clients is two. If I am not planning on the machines doing anything but cado-nfs, would it be more efficient to customize the threads based on the available processors to possibly one client using all threads? I could easily do this within my scripts. I might be over thinking this, but what about the difference between machine capability? Let's say a Core2 Quad 2.4GHz (4 proc) vs. an i7 3.4GHz with HT (8 proc). Would it be better to load up a single client on both machines, or maybe give the Core2 a single loaded client and the i7 four, two thread clients for a total of five more balanced instances? How much value does bogomips have? Ed

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
CADO-NFS on windows	jux	CADO-NFS	25	2021-07-13 23:53
CADO help	henryzz	CADO-NFS	4	2017-11-20 15:14
CADO and WinBlows	akruppa	Programming	22	2015-12-31 08:37
CADO-NFS	skan	Information & Answers	1	2013-10-22 07:00
CADO	R.D. Silverman	Factoring	4	2008-11-06 12:35

2018-05-04, 03:18	#188
EdH "Ed Hall" Dec 2009 Adirondack Mtns EE9₁₆ Posts	Curtis, I've added another machine to my mix and hope to rework some of my scripts in the next few days. I'm trying to see if there is a way to fit a 160 into my daytime on time for several of the machines. I think you came up with a doubling of effort at something over 5 digits with cado-nfs. I calculated the poly select and sieving at just over 7 hours for the C150. This only allows for one doubling, if that, within my limited window. How do you think your C150 params would run for a C155, if I try to see how it would compare to the C150, realizing an added machine would skew any direct comparison? Ed

2018-05-04, 04:06	#189
VBCurtis "Curtis" Feb 2005 Riverside, CA 11375₈ Posts	Ed- I'll have a C155 best-guess file for you in a couple hours. I won't change much from the C150, but I'll try to make it a bit better than the existing C150 params would be. If you happen to still have the log from the good C150 run, try to dig through the file to discover the last Q searched? That info gives me an idea of whether the alim/rlim choices need to be inflated a bit for a larger factorization. I'm hoping this final-Q is around 15-18M. If I get the lim's set correctly for C155, you should manage to come in under double the time for C150. I realized that I can make poly select a bit more efficient for your mega-multi-process setup; I'll put that into the params file too. It's a simple change: poly select was previously split into segments of 500, with CADO searching every multiple of 60. If we change that to 480, each process will always search 8 coefficients, instead of some receiving 9 to split into two threads. Fewer single-threaded running will take place, so wall-clock time will improve a bit for the same CPU-time.

2018-05-04, 04:08	#190
wombatman I moo ablest echo power! May 2013 29×61 Posts	Curtis, would you be willing to go ahead and take a stab at a C165 parameter file (or suggest what you think I could change to improve on the default)? I have a C164 I want to tackle with CADO, but given that you were able to cut the time on the C150 in half, I think it would be more than worthwhile to try and do the same for the C164.

2018-05-04, 15:26	#195
VBCurtis "Curtis" Feb 2005 Riverside, CA 4,861 Posts	Excellent! Q=16M is in the optimistic side of the expected range. The only reason to adjust from 2 threads per process that I know of is to save memory. I=15 uses something on the order of 2GB per process, but the thread pool shares that footprint; a 4GB 4-core machine would want to run 4 threads on a single client rather than two on two clients to save memory when I=15 (Say, GNFS-180+). I believe I=14 uses around 500MB per client, which should be no problem even with multiple clients running.

2018-05-05, 12:43	#198
VictordeHolland "Victor de Hollander" Aug 2011 the Netherlands 2³×3×7² Posts	Does anybody got CADO-NFS working under windoz? On Ubuntu it is super easy with "git clone" and "make" and it will get all the packages it needs. I compiled GMP, ECM and mlucas on Windows using Mingw, but this is a totally different beast. Even with the instructions included in the git repo :(. But I'm still a newbie with compiling....