View Single Post
Old 2021-07-11, 04:57   #19
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2×1,789 Posts
Default

As best as I can tell, here is the situation:

Msieve poly select has two deadlines, one per coefficient and one per job. The coefficient deadline is huge, 100 days or something, so if a coefficient gets stuck it will stay stuck until the job deadline expires. I think that is why some threads get lazy, as you say... they are waiting for the job deadline to expire. Prior to the latest change I made, that deadline was msieve's internal job deadline, which for c120's is 4 hours or so and for c156 I think 300+ hours. Yafu's "fast" mode divides the job deadline by the number of threads, but msieve itself isn't threaded and doesn't know anything about this "fast" deadline so if a coefficient gets stuck it still stays stuck until the 300 hour (for example) job deadline expires. At least that's what I think was happening.

In the most recent change for yafu (made a couple weeks ago or so), I was able to modify msieve's job deadline to match yafu's divided down "fast" deadline, so if a coefficient gets stuck it should expire close to the same time as the rest of the threads. At the time I ran a bunch of tests on C110's and C120's with lots of threads and things worked much better. This is yafu 2.03, although that version number may not have changed for every update (I am not so good at remembering to do that every time). So bur's 2.03 may or may not have that change in it.

One thing I have mentioned already, and which I suspect bur might be seeing too, is that for some input sizes the quality demands in msieve's poly select seem very strict. I have run 80-thread poly select for 15 minutes or more on smallish numbers (don't remember exactly but less than c120), i.e. dozens of thread hours, without msieve reporting a single poly it thinks is good enough, even though there are hundreds of polys below its threshold that will take that 80-thread monster cpu 10 minutes to sieve. I am not trying to complain, and I would like to be able to help, but I haven't and won't have time to figure out better numbers for msieve to use. For most size inputs it is fine... but for other sizes it seems really strict. I suspect most people don't even notice because most people seem to use gpu's or cado now.

Long story short: the newest yafu is trying to get better at this for those of us (me) that still only use msieve's cpu poly select, but I am of course still heavily leaning on msieve which has all of the actual smarts and does all of the actual work. LaurV your idea to have a ctrl-C during poly select do a consolidation and save the best current poly is a good one, I can look into that.
bsquared is offline   Reply With Quote