20210517, 14:42  #23  
Apr 2020
467 Posts 
Quote:
My hunch is that raising tasks.sieve.mfb1 from 62 to 64 would be more effective than using 3LP here. That said, I've got a better idea of what to do with the other parameters than the last time I tried 3LP on a number of roughly this size (see post #2 in this thread). May give it a go in the near future. 

20210518, 07:58  #24  
Aug 2020
5*10398e4;3*2539e3
110001001_{2} Posts 
Thanks for the explanations, slowly I start to get more about what is going on. Interesting, that fundmental things like poly selection or qrange are so little understood that it makes an experimental attempt useful. From the little I could get from math papers, especially poly selection could probably be vastly optimized if we knew more about it.
Quote:
And just to make 100% sure: more relations = faster matrix generation & higher matrix density = smaller matrix size = shorter LA times (not the newspaper, harhar)? 

20210518, 14:43  #25 
"Curtis"
Feb 2005
Riverside, CA
4,951 Posts 
No, there's a flag like filter_maxrels that lets you control the number of relations msieve uses.
TD 100 vs TD 110 shows you the effect of using higher target densities for a given set of relations the difference is not very big. A higher density matrix is slightly tougher to solve than a lowerdensity matrix of the same size, but matrix size affects timetosolve much more. More relations > smaller matrix.> shorter LA time. Higher TD > makes sure you have enough relations for a reasonably small matrix. When filtering with msieve, let's say you have 190M relations. TD = 90 might build a matrix 11M in size, while TD 110 fails and asks for more relations. If you use the full 220M relations, TD 90 might build a 7M matrix, while TD 110 builds a 6.5M matrix. The idea is that choosing TD = 110 makes sure you avoid the situation where a matrix barely builds and is quite large (e.g. 11M for this current factorization). If you actually try these tests, let the matrixsolving step run for 1% completion, and note the ETA (ETA is pretty stable by the time 1% is done). The TD100 and TD110 ETAs won't be very different, I expect. 
20210518, 15:00  #26 
Aug 2020
5*10398e4;3*2539e3
3×131 Posts 
What I meant was, if I only want to test various numbers of relations, which value for TD should I chose, 100? Or is there a default value?
Testing different target_densities was that just a proposal for me to see the effects, or are the results helpful to you similar to the various numbers of relations? 
20210518, 16:27  #27 
"Curtis"
Feb 2005
Riverside, CA
11527_{8} Posts 
Ah, I see yes, I'd use TD 100 to test various relations counts, because higher TDs may not produce matrices. If TD 100 doesn't produce a matrix, that relations count is too low to be useful to us anyway (I think).
The TD 100 vs 110 is useful to us, but I agree it's more for your personal experience. I haven't done as many such tests as I should, so I am curious just how different the ETAs would be; my own limited testing showed the ETAs are similar enough that I rarely use TD 120 even on jobs I expect would easily build a matrix at that density, since the time spent when filtering fails feels greater than the time saved by using 120 instead of 110. 
20210519, 07:02  #28 
Aug 2020
5*10398e4;3*2539e3
3×131 Posts 
I'm still running the tests, results so far show that 185M relations was too few. 190M still worked, LA took from around 14 h (190M) to 8:45 h (220M). Since sieving took 6 1/2 days, doing only 190M would have been quite a gain.
Would it be good to decrease rels_wanted to 190M generally or might this have been an exception? But even 23 additional filtering steps would still make the total time shorter. I will post detailed results later this day. 
20210519, 17:12  #29 
Aug 2020
5*10398e4;3*2539e3
3×131 Posts 
I used this command line ./msieve i c165.n s rels.dat nf c165.fb t 10 v nc1 "filter_maxrels=200000000" afterwards the same with nc2.
220M rels Code:
matrix is 6441919 x 6442144 (2334.4 MB) with weight 618273835 (95.97/col) sparse part has weight 547526160 (84.99/col) [...] linear algebra completed 64760 of 6442144 dimensions (1.0%, ETA 8h43m Code:
matrix is 6671510 x 6671738 (2418.3 MB) with weight 640096824 (95.94/col) sparse part has weight 567230375 (85.02/col) [...] linear algebra completed 66759 of 6671738 dimensions (1.0%, ETA 9h28m Code:
matrix is 6881379 x 6881605 (2496.6 MB) with weight 660645774 (96.00/col) sparse part has weight 585663806 (85.11/col) [...] linear algebra completed 72597 of 6881605 dimensions (1.1%, ETA 10h 3m) Code:
matrix is 7102309 x 7102534 (2579.8 MB) with weight 682640424 (96.11/col) sparse part has weight 605254018 (85.22/col) [...] near algebra completed 71317 of 7102534 dimensions (1.0%, ETA 10h53m) Code:
matrix is 7405037 x 7405262 (2691.5 MB) with weight 711821792 (96.12/col) sparse part has weight 631510830 (85.28/col) [...] linear algebra completed 74878 of 7405262 dimensions (1.0%, ETA 11h54m) Code:
matrix is 7779754 x 7779979 (2833.7 MB) with weight 749219145 (96.30/col) sparse part has weight 665027723 (85.48/col) [...] linear algebra completed 77887 of 7779979 dimensions (1.0%, ETA 13h25m) Code:
matrix is 8305557 x 8305782 (3029.3 MB) with weight 800256398 (96.35/col) sparse part has weight 711056605 (85.61/col) [...] linear algebra completed 87238 of 8305782 dimensions (1.1%, ETA 15h43m) Code:
found 338230 cycles, need 3940439 too few cycles, matrix probably cannot build filtering wants 1000000 more relations I didn't specify TD, which numbers of relations should I chose for the density comparisons? 
20210519, 19:03  #30 
Just call me Henry
"David"
Sep 2007
Cambridge (GMT/BST)
2×3×5×197 Posts 

20210519, 19:18  #31 
"Curtis"
Feb 2005
Riverside, CA
4,951 Posts 
Your comparison of 3.5 hrs per 5M rels shows that any oversieving past 190M is inefficient. This surprises me I would have expected 195 or 200 to be the "sweet spot", where the sieve time spent would be more than or equal to LA time saved.
But, even 195M vs 190M only saves 2hr 20 min LA time at the cost of ~3.5hr sieve time. If these were all default density, I wonder whether the 96 or the 85 density is the target number I don't have msieve logs handy to look it up myself, so I'll edit this message later today to correct myself about which of the densities from your logs is the one msieve controls directly. If default is 96 at this size, I doubt TD is going to do much; try TD 100 or 104 or 110 on the 195M relations set to see if you can save another hour of LA time. Your data also helps someone like Ed, who uses a large farm of ~20 machines to sieve, but just one to LA. Your data at 190.....220 can help him choose how much to sieve. I think I'll make the params file 190M target; as you say, filtering more than once isn't a big deal if that turns out to not be enough for a C161 or C162. I'll add a note that recommends 200M if sieving on multiple machines. What CPU architecture is used for these tests? Older CPUs take relatively longer to do the LA section, so might prove to be 3.5hr faster at 195M because the entire LA portion is taking 50% longer. Sieving is less sensitive to CPU architecture than LA. My personal experience is with sandy bridge, ivy bridge, haswell generations of Xeon, all rather old by now. 
20210519, 22:55  #32  
Apr 2020
467 Posts 
Quote:
Code:
weight of 9242904 cycles is about 924731720 (100.05/cycle) Re general c165 params: lpb 31/31 is definitely worth testing too, unless you already have data showing 31/32 is faster. I also notice you didn't include adjust_strategy 2, which would probably give a boost of a couple percent. Strategy 2 makes A=28 usable but I don't think it'll be optimal yet at this size. Maybe by c170. 

20210520, 07:45  #33  
Aug 2020
5*10398e4;3*2539e3
3×131 Posts 
Quote:
It would make 28 tests, but since filtering is singlethreaded I could run 10 simultaneously. Would this work with all processes accessing the same rels.dat? VBCurtis, it was run on a i1010900K with 16 GB RAM. Nothing else demanding was running at the same time. BTW, I am just factoring a C147 with your experimental params.c145, should I keep the relations to do similar tests? Last fiddled with by bur on 20210520 at 07:47 

Thread Tools  
Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
Some CADONFS Work At Around 175180 Decimal Digits  EdH  CADONFS  127  20201007 01:47 
Sigma parameter in ecm  storm5510  Information & Answers  4  20191130 21:32 
PrimeNet error 7: Invalid parameter  ksteczk  PrimeNet  6  20180326 15:11 
Parameter Underestimation  R.D. Silverman  Cunningham Tables  14  20100929 19:56 
ECM Work and Parameter Choices  R.D. Silverman  Cunningham Tables  11  20060306 18:46 