mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2016-12-25, 14:06   #1
2147483647
 
Dec 2016

3×5 Posts
Default Inefficient behaviour in yafu when doing large NFS with lots of threads

I'm using yafu to do SNFS on some large numbers (about 200 digits) with 48 threads for sieving, and I've noticed that yafu starts 48 sievers but then doesn't start any more until all 48 have finished. This is a problem because some sievers finish after 5 minutes, almost all of them finish after about 7 minutes, but there are usually 1 or 2 processes that don't finish until about 10 minutes. During these 3 minutes, only these 1 or 2 processes are actually doing anything, and the other cpu cores are not being used at all, so the sieving is taking much longer than it should do.

Has anyone else dealt with this issue? Would this be simple to fix?
2147483647 is offline   Reply With Quote
Old 2016-12-25, 14:28   #2
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

2×3×1,699 Posts
Default

Quote:
Originally Posted by 2147483647 View Post
Has anyone else dealt with this issue? Would this be simple to fix?
FactMsieve has the same problem (I don't use yafu myself) so for big factorizations I use a home-grown client-server harness called cabald/cabalc. The name is a relic of a collaboration running in the 90's when RSA-140 was the bleeding edge. It has been used elsewhere, most recently (AFAIK) at EPFL for part of the RSA-768 factorization

One machine runs a cabald daemon which hands out tasks to an arbitrarily large number of cabalc clients. An arbitrary number of cabalc clients can run on each machine, including that which runs cabald.

The harness is very simple and handles only the allocation and processing of sieving tasks. All other duties are up to you, including polynomial finding and the consolidation of relation files for subsequent linear algebra and square root extraction. I normally use factMsieve.pl for all those tasks.

Source code, basic documentation and very limited hand-holding available on request. You are assumed to know what you are doing when running a NFS factorization so assistance is limited to the cabald{c,d} software alone.

Another alternative is CADO, which is a much more professional and powerful client/server version of NFS. It is also complex to drive and the siever is markedly slower than ggnfs. However, once set up it gives a fire-and-forget interface more akin to yafu and factMsieve.pl in that very little human attention is required.
xilman is offline   Reply With Quote
Old 2016-12-25, 16:38   #3
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

13·257 Posts
Default

Quote:
Originally Posted by 2147483647 View Post
I'm using yafu to do SNFS on some large numbers (about 200 digits) with 48 threads for sieving, and I've noticed that yafu starts 48 sievers but then doesn't start any more until all 48 have finished. This is a problem because some sievers finish after 5 minutes, almost all of them finish after about 7 minutes, but there are usually 1 or 2 processes that don't finish until about 10 minutes. During these 3 minutes, only these 1 or 2 processes are actually doing anything, and the other cpu cores are not being used at all, so the sieving is taking much longer than it should do.

Has anyone else dealt with this issue? Would this be simple to fix?
It is a known issue, yes, and I have a fix planned. Not sure when I'll get around to it but hopefully soon.
bsquared is offline   Reply With Quote
Old 2016-12-25, 21:44   #4
yoyo
 
yoyo's Avatar
 
Oct 2006
Berlin, Germany

3×197 Posts
Default

Some of my users blaming the same that many cores are idle.
yoyo is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
POST LOTS AND LOTS OF PRIMES HERE Kosmaj Riesel Prime Search 1955 2020-11-12 15:44
Using Yafu to factor a large number jibanes YAFU 61 2018-07-12 23:11
Strange YAFU behaviour ThomRuley YAFU 7 2012-07-14 04:24
POST LOTS AND LOTS AND LOTS OF PRIMES HERE lsoule Riesel Prime Search 1999 2010-03-17 22:33
lots of large primes Peter Hackman Factoring 2 2008-08-15 14:26

All times are UTC. The time now is 01:20.

Thu Nov 26 01:20:06 UTC 2020 up 76 days, 22:31, 3 users, load averages: 2.46, 1.96, 1.80

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.