mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2020-05-31, 19:51   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

1101001111002 Posts
Default Just a Curiosity

I recently rebuilt msieve and YAFU to fix a crash and noticed this:
Code:
05/31/20 15:36:25 v1.35-beta @ math27, System/Build Info: 
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.1.2
detected        Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
detected L1 = 32768 bytes, L2 = 3145728 bytes, CL = 64 bytes
measured cpu frequency ~= 2494.385440
When I ran "tune:"
Code:
best exponential fit is y = 0.492662 * exp(0.0994914 * x)
QS/NFS crossover occurs at 100.3 digits
Adding tune_info entry for LINUX64 -        Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
Tune info in yafu.ini:
Code:
tune_info=       Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz,LINUX64,2.54497e-05,0.197911,0.492662,0.0994914,100.294,2494.37
When I ran ./yafu via a Perl script:
Code:
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
Edit: I also just noticed this:
Code:
. . .
nfs: commencing polynomial search over range: 17692 - 17942
 deadline: 8640000 CPU-seconds per coefficient
error generating or reading NFS polynomials
nfs: commencing polynomial search over range: 17942 - 18192
deadline: 8640000 CPU-seconds per coefficient
coeff 18060 specialq 1 - 946643 other 6855 - 16452
aprogs: 427 entries, 1206 roots
hashtable: 1024 entries,  0.02 MB
nfs: commencing polynomial search over range: 18192 - 18442
deadline: 8640000 CPU-seconds per coefficient
error generating or reading NFS polynomials
. . .

Last fiddled with by EdH on 2020-05-31 at 19:57
EdH is offline   Reply With Quote
Old 2020-06-15, 15:49   #2
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2·17·97 Posts
Default

I'm seeing normal usage of tune info after testing just now:

Code:
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
Make sure that yafu.ini doesn't have lines containing "xover=" or "snfs_xover=" in it. Yafu will also try to verify that the computer/OS info matches between the current CPU and the tune info text. Based on what you posted they should match, so if it doesn't turn out to be a "xover=" line then I'm not sure what's going on.

I see the same "error generating or reading NFS polynomials" messages during poly search, but it appears to generate polynomials and complete NFS factorizations just fine. I'll file that as low priority for now.

In related news, after the new SIQS updates tune() is now showing a crossover of 106.6 digits for LINUX64! (using a skylake-x.) One should take that number with a large grain of salt given all the assumptions tune() makes, but still, that's higher than I've ever seen it for 64-bit linux ggnfs.

I ran the same C100 through both routines and got 641 seconds for SIQS and 681 for NFS, so 106.6 digits is probably a little optimistic toward SIQS. I'll have to re-visit some of the assumptions tune() makes. (I may have shortened poly select time but not reflected that in tune()).
bsquared is offline   Reply With Quote
Old 2020-06-15, 18:45   #3
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

22×7×112 Posts
Default

Quote:
Originally Posted by bsquared View Post
. . .
Make sure that yafu.ini doesn't have lines containing "xover=" or "snfs_xover=" in it. Yafu will also try to verify that the computer/OS info matches between the current CPU and the tune info text. Based on what you posted they should match, so if it doesn't turn out to be a "xover=" line then I'm not sure what's going on.
I remember leaving those lines in several of my .ini files. These are probably those machines.

I have noticed the latest .ini version you've created. I haven't delved into the inner portions, but is there a reason that "threads" is commented (%) out, as well as being set to 1?

I've been using your msieve modification to cure segfaults on my GMP 6.2.0 machines. Thanks!

And, thanks for all else, as well.
EdH is offline   Reply With Quote
Old 2020-06-15, 18:51   #4
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2·17·97 Posts
Default

Quote:
Originally Posted by EdH View Post

I have noticed the latest .ini version you've created. I haven't delved into the inner portions, but is there a reason that "threads" is commented (%) out, as well as being set to 1?
In my mind, the .ini file should be used to set *permanent* parameters. Things that you always need (for example the ecm path, ggnfs binary folder, and ecm pretesting preferences). Anything in the .ini file can be overridden on the command line. For me, threads is something I frequently change, depending on what I'm trying to do or what networked system I'm currently running on, so I always set that from the command line. Just my preference: customize the .ini as you see fit. The new .ini is simply meant to document the possible options.
bsquared is offline   Reply With Quote
Old 2020-06-15, 18:59   #5
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

D3C16 Posts
Default

Quote:
Originally Posted by bsquared View Post
In my mind, the .ini file should be used to set *permanent* parameters. Things that you always need (for example the ecm path, ggnfs binary folder, and ecm pretesting preferences). Anything in the .ini file can be overridden on the command line. For me, threads is something I frequently change, depending on what I'm trying to do or what networked system I'm currently running on, so I always set that from the command line. Just my preference: customize the .ini as you see fit. The new .ini is simply meant to document the possible options.
I just remembered something else to bring up, but will have to wait for another tune session to show an example. I get a repetitive skewed error message with the new .ini file. All seems to run fine afterward, though. This was with and without removing the existing tune data. I suppose it might have to do with the threads edit. I'll post an example later, unless it never shows again.
EdH is offline   Reply With Quote
Old 2020-06-15, 19:04   #6
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2×17×97 Posts
Default

Quote:
Originally Posted by EdH View Post
I just remembered something else to bring up, but will have to wait for another tune session to show an example. I get a repetitive skewed error message with the new .ini file. All seems to run fine afterward, though. This was with and without removing the existing tune data. I suppose it might have to do with the threads edit. I'll post an example later, unless it never shows again.
I saw this too. I think it has to do with blank lines in the .ini. I changed the ReadINI function to ignore these but the writeINI function that is called as part of tune did not get changed and doesn't like them. In mine the blank line were removed, in addition to adding the tune_info line, as an after effect of running tune(). It seemed Mostly Harmless (TM), but I will try to fix at some point anyway.

If you are referring to something else let me know.
bsquared is offline   Reply With Quote
Old 2020-06-15, 21:28   #7
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

22×7×112 Posts
Default

Quote:
Originally Posted by bsquared View Post
I saw this too. I think it has to do with blank lines in the .ini. I changed the ReadINI function to ignore these but the writeINI function that is called as part of tune did not get changed and doesn't like them. In mine the blank line were removed, in addition to adding the tune_info line, as an after effect of running tune(). It seemed Mostly Harmless (TM), but I will try to fix at some point anyway.

If you are referring to something else let me know.
I'm pretty sure that's what it was. The blank line was removed in the one I just ran and there was no error - all looked correct.
EdH is offline   Reply With Quote
Old 2020-06-16, 01:15   #8
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

22·7·112 Posts
Default

Yep, an extra blank line (or two) prior to the previous tune info (and 100.1 digit crossover:)
Code:
QS/NFS crossover occurs at 100.1 digits
Invalid line in yafu.ini, use Keyword=Value pairsSee docfile.txt for valid keywords
. . .<38 copies of these lines>. . .
Invalid line in yafu.ini, use Keyword=Value pairsSee docfile.txt for valid keywords
found OS = LINUX64 and CPU = Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz in tune_info field
Adding tune_info entry for LINUX64 -        Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz

Last fiddled with by EdH on 2020-06-16 at 01:17 Reason: repare speelin
EdH is offline   Reply With Quote
Old 2020-06-16, 01:53   #9
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

22×7×112 Posts
Default

I think your larger crossover might explain the results of an experiment I've been conducting:

Two machines "virtually" identical.

Machine 1: Aliqueit running YAFU for qs, ecm.py for ECM and CADO-NFS for GNFS

Machine 2: Aliqueit running option -y (YAFU for all)

Both were started on an existing sequence that was around 105 digits.

Although Machine 1 seemed to be doing a lot more ECM than Machine 2, it ran away, leaving machine 2 many lines behind. At this time, the sequence had taken a downturn and was shedding digits.

I swapped the setups and copied the longer sequence to Machine 2, so they would start even again. This time Machine 1, running YAFU (-y) ran away by over 30 indices, at first! As the digits rose again, Machine 2 caught up with and passed Machine 1, even though it had been 30 indices behind.

I will have to do some more comparisons to really conclude anything, but the preliminary shows CADO-NFS to be promising. As to my using CADO-NFS, for my Aliqueit runs, I'm implementing distributed polyselect and sieving across >20 machines of various architecture. I was able to do this with factmsieve.py, too, but CADO-NFS is simpler and now it seems faster, also.

I noticed a query as to the possibility of YAFU using CADO-NFS and know this would be quite a bit of work, but from my playing around and other tests, etc., if you ever do consider CADO-NFS, I would suggest still letting Msieve do the LA.
EdH is offline   Reply With Quote
Old 2020-06-16, 02:18   #10
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

440710 Posts
Default

Quote:
Originally Posted by EdH View Post
..... if you ever do consider CADO-NFS, I would suggest still letting Msieve do the LA.
I think on small inputs the difference in LA time isn't enough to be worth much programming effort, since it's more complicated to use a hybrid versus cado-as-black-box-factorer. I don't bother to break CADO to invoke msieve for LA below 150 digits, and by that size lots of folks aren't using YAFU anyway. msieve may be 2/3rds the time, but the matrix time for a 130 digit number just isn't that long.
VBCurtis is online now   Reply With Quote
Old 2020-06-16, 11:51   #11
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

64748 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
I think on small inputs the difference in LA time isn't enough to be worth much programming effort, since it's more complicated to use a hybrid versus cado-as-black-box-factorer. I don't bother to break CADO to invoke msieve for LA below 150 digits, and by that size lots of folks aren't using YAFU anyway. msieve may be 2/3rds the time, but the matrix time for a 130 digit number just isn't that long.
But, if the programming takes care of the hybridization, why not try for the fastest factoring possible? I would think if you already have msieve doing LA and you want to inject CADO into a package for polyselect and sieving, it would be best to just inject that faster part. Why cripple LA, even a little?
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Results Curiosity Gordon PrimeNet 1 2015-07-30 06:49
I see this as a bit of a mathematical curiosity... petrw1 Math 4 2015-07-19 02:33
A Curiosity R.D. Silverman GMP-ECM 4 2012-05-10 20:00
P-1 question of curiosity JuanTutors Math 3 2004-10-20 04:37
Curiosity ET_ Math 7 2004-02-13 17:36

All times are UTC. The time now is 05:08.

Fri Oct 30 05:08:01 UTC 2020 up 50 days, 2:18, 1 user, load averages: 2.28, 2.15, 2.10

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.