CADO NFS

[VBCurtis]

Originally Posted by EdH

Thanks - worked great! Is there some way to tell if I will have this trouble prior to translating the poly file to the fb file. If I can tell at that time, I can translate as needed. Should c0/A0 be positive to start with, perhaps?

If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.

[VBCurtis]

Originally Posted by VictordeHolland

\5569 seconds WCT (1:32:49). GGNFS was faster, but only 1-2 minutes!! Also, CADO continued sieving while attempting filtering, while the GGNFS/factmsieve.py script waits for the outcome of the filtering attempt.
And look at the E scores: e 3.305e-10 vs. Murphy_E = 2.47e-09
Did CADO just find a really, really good poly with (1 - 100 000)??? Or did GGNFS/Msieve somehow fail (or did I somewhere??)
program

Please tell me what did I do wrong? There must be some mistake somewhere or the believe that GGNFS/Msieve/factmsieve.py package is faster for small composites needs revision?

CADO calculates E-score differently; the numbers don't compare directly for poly scores. CADO calculates the score based on the sieving region you declare in the params file; if you change lim0 and lim1 the reported poly score will change also. This is more accurate for comparing polys (using the params you intend to sieve on rather than a standard-candle approach), but it means scores aren't comparable to msieve scores.

As for your timings, I believe CADO handles many-threads better than factmsieve/msieve. If you repeat your RSA100 experiment with only 4 threads for each package, I think msieve wins. That's what has been reported here time and time again, though some of those reports preceded my improved params files.

CPU time is, to me, a better measure than wall-clock time; as you noted factmsieve filters single-threaded and the sievers don't run during that time. I often run small factorizations on the "free" hyperthreads on a 6-core machine, so any idle time in factmsieve is being used by other processes running on the machine.

[Dubslow]

Originally Posted by VBCurtis

CADO calculates E-score differently; the numbers don't compare directly for poly scores. CADO calculates the score based on the sieving region you declare in the params file; if you change lim0 and lim1 the reported poly score will change also. This is more accurate for comparing polys (using the params you intend to sieve on rather than a standard-candle approach), but it means scores aren't comparable to msieve scores.

It's also problematic when the sieve parameters aren't known until after poly select is complete, e.g. adjusting lpb's depending on how good a poly is found, testing degree transitions, gnfs vs snfs testing, etc... my overall experience with the CADO package is that, while the networking and individual subcomponents are great, the glue python code is extremely monolithic and inflexible. The entire process start to finish must be done one way only, and any variations on that process (such as are quite common on this forum for distributed projects, such as team sieves or multiperson poly searches etc) will be ~impossible. A couple years ago when I was playing with the poly select code I wound up writing my own python wrappers around the C executables to be able to do things like polsel stage 1+sizeopt separately from rootopt. (And they are indeed different executables! But looking at CADO's python you'd never know that.)

(And, btw in reference to the rest of your post, is CADO's filtering not single threaded? I had assumed it was, is my assumption incorrect?)

[EdH]

Originally Posted by VBCurtis

If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.

I wrote before thinking - not uncommon** for me.

Thanks!

**Much of the time I catch it in my proofreading, but shorter posts proofread more poorly...

[jasonp]

Originally Posted by VBCurtis

If you mean c5/a5, yes. It's the leading coeff that msieve doesn't like.

Msieve can handle leading rational coefficients that are negative without issue, I just wasn't sure how the arithmetic in the square root changed when the leading algebraic coefficient was negative. For all I know it would work fine without changes, or would work fine on average, but I didn't want to take that risk when there is already so much that can go wrong in the square root.

[henryzz]

Originally Posted by jasonp

Msieve can handle leading rational coefficients that are negative without issue, I just wasn't sure how the arithmetic in the square root changed when the leading algebraic coefficient was negative. For all I know it would work fine without changes, or would work fine on average, but I didn't want to take that risk when there is already so much that can go wrong in the square root.

Assuming that inverting the sign always works it might be worth msieve always doing that with a negative leading algebraic coefficient.

[VBCurtis]

You'd think I would not need to discover this twice:
If you run CADO within "screen", and the session is not live when the factorization completes, CADO makes no record of the results whatsoever. The temp folder with all the data is erased, including the log, and since the process finished "screen -r" has nothing to resume.
So, I have only very rough wall-clock data for the C149 I ran.
On the bright side, I decided to try Ed's relation-catting-for-msieve, so I have all the relations and msieve is matrix-ing now. At least I get factors!

[Dubslow]

I didn't even know you could use screen just for one process lol. I just run screen, and then once in the screen whatever command I need, and it always persists until I close it.

[VBCurtis]

I have a helpful ISP that drops any connection after 30 min of inactivity. At least, I think it's the ISP.... anyway, my server is at work and I SSH in to do my factoring jobs. Most jobs have screen activity often enough to not matter, but msieve filtering and CADO sqrt both cause disconnections.
"screen python ./factmsieve.py [etc]" means the job finishes and all is well; doing the same with CADO is a problem.
I reported this as a bug, requesting a file be written (or even the params file be edited) with factors found, and was told I could fix it myself by telling CADO to not use /tmp for its work.
I'm just not a fan of any program that only outputs results to screen.... perhaps I'll start making my own log file by redirecting screen output to a file.

Also, on this C149 candidate, CADO took ~31 hr for the matrix, while msieve ETA is 18 hr total (after 10% has run, so ETA should be accurate). Sieving on 12 threads of a 12-core Sandy Bridge system doing ~11 other things (7xLLR, 3x ECM, srsieve) took just under 96 hr.
I tried msieve postprocessing on the relation set at 92% of CADO's targeted relations, without success. CADO ended up sieving to about 105% of its own targeted relations, and that set produced a 4.4M matrix in msieve without setting Target Density. I believe CADO said its matrix was 3.8M, suggesting my CADO-params file has target density set higher than msieve-equivalent-70 (I think that's a plus, given how slow CADO solves matrices).

[RichD]

So what happens when you get a power outage after a factorization completes in the middle of the night during a thunderstorm? Oh, don't get me started about weather related power issues in the Midwest.

[Gimarel]

The comparison was run on an i7-3770K @ 4GHz.

YAFU/msieve with patches for better parametrisation
Poly selection: 02:21:19
Sieving: 41:05:55
Matrix: 01:45:36
Total: 45:45:00

CADO with default parameters
Poly selection: 01:58:30
Sieving: 95:00:34
Matrix: 14:13:40
Total: 112:04:07

Total time includes setup, filtering, and square root.
Note that the CADO poly had a msieve score that was about 5% better than the score of the YAFU poly.