[QUOTE=jyb;501725]What? Here's what you said in the other thread:



And yes, I know you later say that it should be "3.7 or 3.8 or maybe 3.9" for 3LP, but it doesn't seem like you can give a reason why. And while I am sometimes willing to do things simply because they are considered best practices, nobody seems to have any actual understanding of what the best practice is here and why. So while I certainly don't know better than you here, I'm not terribly inclined to make this change without some further explanation.

Edit: And by the way, saying "I don't really understand why, but this will speed up sieving by 10%" would be a perfectly fine explanation for these purposes. But I haven't heard anything to that effect, or heard tell of any evidence that something like that should be the case.[/QUOTE]
Sorry for being unclear. My comments about 2.0 and 3.0 were meant as evidence that my first explanation of lambda couldn't be correct, because both standard practice and test-sieving shows that 2.5 or 2.6 is always better than 2.0 or 2.1; the idea I tried and failed to convey is that using 3LP means adding 1 to that side's lambda. If you accept that 2.6 works for normal 2LP jobs, please accept that 3.6 is what works for 3LP jobs.

Or, as axn says, test-sieve a bunch of lambdas and let us know what you think. But be warned that test-sieves will show different effects at different Q values; it's messy, or else we'd have a clear best choice already.

[QUOTE=VBCurtis;501754]Or, as axn says, test-sieve a bunch of lambdas and let us know what you think. But be warned that test-sieves will show different effects at different Q values; it's messy, or else we'd have a clear best choice already.[/QUOTE]

Okay, well here's some data for the last number I posted (HCN 7+6,320):
[code]
2.6 3.0 3.6
--- --- ---
Q Yield sec/rel Yield sec/rel Yield sec/rel
--- ----- ------- ----- ------- ----- -------
20M 5235 0.20297 5264 0.20422 11022 0.13363
50M 7521 0.15008 7540 0.15242 12750 0.14660
80M 8104 0.14465 8117 0.14794 12003 0.15987
110M 9760 0.14617 9765 0.15100 13317 0.17087
140M 8957 0.15357 8960 0.15806 11798 0.18048
170M 8842 0.15327 8848 0.15776 11572 0.18024
200M 8993 0.15947 8998 0.16470 11802 0.18607
230M 8577 0.16492 8585 0.16982 11204 0.19071
[/code]

As you can see, a lambda of 3.0 isn't appreciably different from 2.6. But moving it to 3.6 changes things substantially. And as you said, the effects are quite different for different Q values. Across the whole range, yields are quite a bit higher, but completing a given range is quite a bit slower. And for purposes of assessing total speed of sieving, these two effects run counter to each other. At lower Q values, the speed is greater (very much so for Q around 20M), while at higher Q values it's worse than using a lambda of 2.6 or 3.0.

For determining which is the better choice for this particular number, I note that the higher yield with lambda=3.6 means having to sieve a smaller range (probably just up to about 120M), so we'd spend more of the time in the Q ranges where that higher lambda gives us better speed. Still, total sieving time probably ends up being fairly close, regardless of which value we choose.

However, when thinking about this in terms of NFS@Home, the practical effect of using the higher lambda is that the smaller range means fewer work units get generated, and each one takes longer to complete. That cuts down on the parallelism that we get from having thousands of cores doing the sieving. So the higher lambda may slow down the completion of sieving this number. More generally, if other numbers have the same characteristics, then we get individual numbers potentially taking longer, but only because they are handed out to fewer cores. I.e. it increases the latency, but it should not affect the throughput of the grid as a whole.

On the other hand, I don't know what effect the higher lambda has on eventual matrix size. My intuition is that the smaller range of sp-Q means a smaller matrix, but I don't have enough experience to trust my intuition on this point, so someone please weigh in. But if that's true, then I would argue it's a tradeoff worth making. I.e. increasing the latency of each number, but also decreasing post-processing time seems like a fine tradeoff, and all the more so if it doesn't hurt throughput.

Any more knowledgeable thoughts and opinions are welcome. (Less knowledgeable ones are also welcome, but being more knowledgeable than myself is a low bar.)

The similarity in yields between 2.6 and 3.0 tells you that 3.0 isn't actually using 3 large primes. The yield from 3.6 looks like the usual result of 3LP: much higher yield, and sec/rel that increases quite a lot more with Q than it does with 2LP. I'm quite interested to compare 3.6 to 3.7 to either.3.5 or 3.8; perhaps I'll do so next time I test-sieve for 15e queue.

I did a bit of comparing jobs in my 13*2^n-1 factoring personal project, running some with 3LP and other similar-difficulty jobs with 2LP. It seemed that 3LP produces matrices slightly larger than 2LP, such that I wanted to see a clear win in sec/rel for 3LP in order to use it. I didn't see that clear win until roughly SNFS-250+; on GNFS jobs I haven't done enough work to have a good estimate (I would guess 183 or 185 for 15e queue, but it's lower for CADO).

59999_216 Needs its sieve upper boundary raised by 15-20M.

[B]QUEUED AS 4p3_445[/B]

SNFS-214.3 (quartic) C170 HCN (4+3,445), ECM to t53+. For 14e.
[code]
n: 56762202779198031324149362229206360783071902215738462242361981308944956677664820201689887498819757592579638106202107993072829500335001135923337125601798258607407132719981
# 4^445+3^445, difficulty: 214.33, skewness: 1.00, alpha: 1.45
skew: 1.000
c4: 1
c3: -1
c2: 1
c1: -1
c0: 1
Y1: -2909321189362570808630465826492242446680483
Y0: 383123885216472214589586756787577295904684780545900544
rlim: 134000000
alim: 134000000
lpbr: 31
lpba: 31
mfbr: 62
mfba: 62
rlambda: 2.6
alambda: 2.6
[/code]
Trial sieving 5K blocks:
[code]
Q Yield
--- -----
20M 6446
50M 9168
80M 9801
110M 11534
140M 10849
170M 10386
200M 10511
[/code]
Recommend sieving special Q on rational side, 20M - 150M.

[B]QUEUED AS 3m2_575[/B]

SNFS-219.5 (quartic) C179 HCN (3-2,575), ECM to t54+. For 14e.
[code]
n: 25937720731707886917929720519505136746571172607323113360630998764400873985091931948840691828554767040626838855185461368087763740650541795343670441758207306840627095008652329080201
# 3^575-2^575, difficulty: 219.48, skewness: 1.00, alpha: 1.45
skew: 1.000
c4: 1
c3: 1
c2: 1
c1: 1
c0: 1
Y1: -41538374868278621028243970633760768
Y0: 7395104114874202511988394360121831439224179537192802907
rlim: 134000000
alim: 134000000
lpbr: 31
lpba: 31
mfbr: 62
mfba: 62
rlambda: 2.6
alambda: 2.6
[/code]
Trial sieving 5K blocks:
[code]
Q Yield
--- -----
20M 4000
50M 5659
80M 6231
110M 7544
140M 6702
170M 6565
200M 6861
[/code]
Recommend sieving special Q on rational side, 20M - 210M.

If you note the relatively low yield and it makes you uncomfortable, ask me for more details of trial sieving. These parameters look to be substantially faster than any of the knob settings I tried for 32 bits.

[QUOTE=jyb;502050]
If you note the relatively low yield and it makes you uncomfortable, ask me for more details of trial sieving. These parameters look to be substantially faster than any of the knob settings I tried for 32 bits.[/QUOTE]

Sir, our last discussion demonstrated to me that you're quite thorough with trial sieving, and are considering as alternatives my concept of "normal". I trust your tests and your judgement.

Your results also tell me I need to test smaller LP bounds on my own submissions; for that, I am grateful.

[QUOTE=jyb;502050]SNFS-219.5 (quartic) C179 HCN (3-2,575), ECM to t54+. For 14e.
[/QUOTE]

Since this is a quartic, and the yields are so low (1.4 / q), this suggests that this should be handled by 15e. However, if 14e can get the job done and "The Grid(TM)" demands that 14e be fed, so be it.

[QUOTE=VBCurtis;502051][COLOR="Red"]Sir[/COLOR] :surprised:, our last discussion demonstrated to me that you're quite thorough with trial sieving, and are considering as alternatives my concept of "normal". I trust your tests and your judgement.

Your results also tell me I need to test smaller LP bounds on my own submissions; for that, I am grateful.[/QUOTE]

Perhaps. I appreciate the vote of confidence, though I'm not entirely so confident myself.

In particular, there may be limits to the predictive value of trial sieving on small ranges. One point on which I know I'm ill-informed is just what kind of effect using more large primes actually has. I vaguely recall getting the impression that a lot of the advantage of large primes doesn't show up until late in the sieving, when there are already a lot of relations gathered. If that's true, then an important effect of large primes is not captured by doing trial sieving, and I have heard no good guidelines on how to estimate that effect.

So while I may choose the set of parameters that suggests the best times during trial sieving, it's still possible that your more "normal" parameters would still lead to a better outcome overall. Any additional education you (or anyone else here) can give me on this point would be welcome.

[QUOTE=axn;502054]Since this is a quartic, and the yields are so low (1.4 / q), this suggests that this should be handled by 15e. However, if 14e can get the job done and "The Grid(TM)" demands that 14e be fed, so be it.[/QUOTE]

Ah, well now this is an interesting point. The last time I messed around with quartics, a couple of years ago, I recall noting that an SNFS difficulty of 225 was the point at which it seemed better to use the 15e siever. However, I know I wasn't exhaustive in my investigation, so it's possible that the crossover point could really be a bit lower.

But in any case, you raise a good issue. I should really add 15e to the mix and see what happens. Of course, trial sieving aside, the lag time before we actually know how well the parameters have been chosen is much greater for 15e because that queue is well-filled, unlike 14e, which is always begging for new candidates.

[B]QUEUED AS 5m4_775M[/B]

SNFS-216.7 (quartic) C198 HCN (5-4,775M), ECM to t53+. For 14e.
[code]
n: 453062901311293760348068354191848950392011558760260537029420469905603330314318999280703940136577954077974134417119697410057865928213719081382707484149074079611650566663498545608756927899481187651901
skew: 0.894427
c4: 25
c3: 50
c2: 60
c1: 40
c0: 16
Y1: 22835963083295358096932575511191922182123945984
Y0: -661744490042422139897126953655970282852649688720703125
rlim: 134000000
alim: 134000000
lpbr: 31
lpba: 31
mfbr: 62
mfba: 62
rlambda: 2.6
alambda: 2.6
[/code]
Trial sieving 5K blocks:
[code]
Q Yield
--- -----
20M 5005
50M 6854
80M 7326
110M 8987
140M 8169
170M 7796
[/code]
Recommend sieving special Q on rational side, 20M - 180M.