mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > CADO-NFS

Reply
 
Thread Tools
Old 2018-05-15, 04:50   #243
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41×109 Posts
Default

Quote:
Originally Posted by EdH View Post
Thanks - worked great! Is there some way to tell if I will have this trouble prior to translating the poly file to the fb file. If I can tell at that time, I can translate as needed. Should c0/A0 be positive to start with, perhaps?
If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.
VBCurtis is offline   Reply With Quote
Old 2018-05-15, 04:58   #244
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41·109 Posts
Default

Quote:
Originally Posted by VictordeHolland View Post
\5569 seconds WCT (1:32:49). GGNFS was faster, but only 1-2 minutes!! Also, CADO continued sieving while attempting filtering, while the GGNFS/factmsieve.py script waits for the outcome of the filtering attempt.
And look at the E scores: e 3.305e-10 vs. Murphy_E = 2.47e-09
Did CADO just find a really, really good poly with (1 - 100 000)??? Or did GGNFS/Msieve somehow fail (or did I somewhere??)
program

Please tell me what did I do wrong? There must be some mistake somewhere or the believe that GGNFS/Msieve/factmsieve.py package is faster for small composites needs revision?
CADO calculates E-score differently; the numbers don't compare directly for poly scores. CADO calculates the score based on the sieving region you declare in the params file; if you change lim0 and lim1 the reported poly score will change also. This is more accurate for comparing polys (using the params you intend to sieve on rather than a standard-candle approach), but it means scores aren't comparable to msieve scores.

As for your timings, I believe CADO handles many-threads better than factmsieve/msieve. If you repeat your RSA100 experiment with only 4 threads for each package, I think msieve wins. That's what has been reported here time and time again, though some of those reports preceded my improved params files.

CPU time is, to me, a better measure than wall-clock time; as you noted factmsieve filters single-threaded and the sievers don't run during that time. I often run small factorizations on the "free" hyperthreads on a 6-core machine, so any idle time in factmsieve is being used by other processes running on the machine.
VBCurtis is offline   Reply With Quote
Old 2018-05-15, 12:40   #245
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3·2,399 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
CADO calculates E-score differently; the numbers don't compare directly for poly scores. CADO calculates the score based on the sieving region you declare in the params file; if you change lim0 and lim1 the reported poly score will change also. This is more accurate for comparing polys (using the params you intend to sieve on rather than a standard-candle approach), but it means scores aren't comparable to msieve scores.
It's also problematic when the sieve parameters aren't known until after poly select is complete, e.g. adjusting lpb's depending on how good a poly is found, testing degree transitions, gnfs vs snfs testing, etc... my overall experience with the CADO package is that, while the networking and individual subcomponents are great, the glue python code is extremely monolithic and inflexible. The entire process start to finish must be done one way only, and any variations on that process (such as are quite common on this forum for distributed projects, such as team sieves or multiperson poly searches etc) will be ~impossible. A couple years ago when I was playing with the poly select code I wound up writing my own python wrappers around the C executables to be able to do things like polsel stage 1+sizeopt separately from rootopt. (And they are indeed different executables! But looking at CADO's python you'd never know that.)

(And, btw in reference to the rest of your post, is CADO's filtering not single threaded? I had assumed it was, is my assumption incorrect?)
Dubslow is offline   Reply With Quote
Old 2018-05-15, 12:52   #246
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

5·13·53 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.
I wrote before thinking - not uncommon** for me.

Thanks!

**Much of the time I catch it in my proofreading, but shorter posts proofread more poorly...

Last fiddled with by EdH on 2018-05-15 at 12:56 Reason: After thoughts
EdH is offline   Reply With Quote
Old 2018-05-15, 16:24   #247
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

1101101000012 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.
Msieve can handle leading rational coefficients that are negative without issue, I just wasn't sure how the arithmetic in the square root changed when the leading algebraic coefficient was negative. For all I know it would work fine without changes, or would work fine on average, but I didn't want to take that risk when there is already so much that can go wrong in the square root.
jasonp is offline   Reply With Quote
Old 2018-05-15, 21:10   #248
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

574310 Posts
Default

Quote:
Originally Posted by jasonp View Post
Msieve can handle leading rational coefficients that are negative without issue, I just wasn't sure how the arithmetic in the square root changed when the leading algebraic coefficient was negative. For all I know it would work fine without changes, or would work fine on average, but I didn't want to take that risk when there is already so much that can go wrong in the square root.
Assuming that inverting the sign always works it might be worth msieve always doing that with a negative leading algebraic coefficient.
henryzz is offline   Reply With Quote
Old 2018-05-16, 01:49   #249
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41·109 Posts
Default

You'd think I would not need to discover this twice:
If you run CADO within "screen", and the session is not live when the factorization completes, CADO makes no record of the results whatsoever. The temp folder with all the data is erased, including the log, and since the process finished "screen -r" has nothing to resume.
So, I have only very rough wall-clock data for the C149 I ran.
On the bright side, I decided to try Ed's relation-catting-for-msieve, so I have all the relations and msieve is matrix-ing now. At least I get factors!
VBCurtis is offline   Reply With Quote
Old 2018-05-16, 02:48   #250
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3·2,399 Posts
Default

I didn't even know you could use screen just for one process lol. I just run screen, and then once in the screen whatever command I need, and it always persists until I close it.
Dubslow is offline   Reply With Quote
Old 2018-05-16, 03:52   #251
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

41·109 Posts
Default

I have a helpful ISP that drops any connection after 30 min of inactivity. At least, I think it's the ISP.... anyway, my server is at work and I SSH in to do my factoring jobs. Most jobs have screen activity often enough to not matter, but msieve filtering and CADO sqrt both cause disconnections.
"screen python ./factmsieve.py [etc]" means the job finishes and all is well; doing the same with CADO is a problem.
I reported this as a bug, requesting a file be written (or even the params file be edited) with factors found, and was told I could fix it myself by telling CADO to not use /tmp for its work.
I'm just not a fan of any program that only outputs results to screen.... perhaps I'll start making my own log file by redirecting screen output to a file.

Also, on this C149 candidate, CADO took ~31 hr for the matrix, while msieve ETA is 18 hr total (after 10% has run, so ETA should be accurate). Sieving on 12 threads of a 12-core Sandy Bridge system doing ~11 other things (7xLLR, 3x ECM, srsieve) took just under 96 hr.
I tried msieve postprocessing on the relation set at 92% of CADO's targeted relations, without success. CADO ended up sieving to about 105% of its own targeted relations, and that set produced a 4.4M matrix in msieve without setting Target Density. I believe CADO said its matrix was 3.8M, suggesting my CADO-params file has target density set higher than msieve-equivalent-70 (I think that's a plus, given how slow CADO solves matrices).
VBCurtis is offline   Reply With Quote
Old 2018-05-16, 04:23   #252
RichD
 
RichD's Avatar
 
Sep 2008
Kansas

23·397 Posts
Default

So what happens when you get a power outage after a factorization completes in the middle of the night during a thunderstorm? Oh, don't get me started about weather related power issues in the Midwest.
RichD is offline   Reply With Quote
Old 2018-05-16, 04:56   #253
Gimarel
 
Apr 2010

24×32 Posts
Default c145 comparison cado vs. yafu/msieve

The comparison was run on an i7-3770K @ 4GHz.

YAFU/msieve with patches for better parametrisation
Poly selection: 02:21:19
Sieving: 41:05:55
Matrix: 01:45:36
Total: 45:45:00

CADO with default parameters
Poly selection: 01:58:30
Sieving: 95:00:34
Matrix: 14:13:40
Total: 112:04:07

Total time includes setup, filtering, and square root.
Note that the CADO poly had a msieve score that was about 5% better than the score of the YAFU poly.

Last fiddled with by Gimarel on 2018-05-16 at 04:59
Gimarel is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
CADO-NFS on windows jux CADO-NFS 22 2019-11-12 12:08
CADO help henryzz CADO-NFS 4 2017-11-20 15:14
CADO and WinBlows akruppa Programming 22 2015-12-31 08:37
CADO-NFS skan Information & Answers 1 2013-10-22 07:00
CADO R.D. Silverman Factoring 4 2008-11-06 12:35

All times are UTC. The time now is 14:13.

Wed Nov 25 14:13:48 UTC 2020 up 76 days, 11:24, 3 users, load averages: 0.94, 1.19, 1.31

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.