mersenneforum.org Parameter explorations for CADO 165-170 digits
 Register FAQ Search Today's Posts Mark Forums Read

 2021-09-02, 15:33 #56 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22·1,249 Posts Each polyselect process runs two-threaded, with each thread working independently. Using adrange of 8 * incr means each thread gets four leading coefficients to search. It is unavoidable that one set of four will finish before the other, so every polyselect process will run single-threaded for some time at the end of its workunit. This is part of the reason why using hyperthreads gains speed with CADO. I haven't changed anything else on params.c165; in fact, I haven't solved a job that big since you did so.
2021-09-02, 17:30   #57
bur

Aug 2020
79*6581e-4;3*2539e-3

39910 Posts

Quote:
 This is part of the reason why using hyperthreads gains speed with CADO.
I didn't know that. For a 10 core should I use 5 clients à 4 threads or 10 à 2? And does sieving also benefit from hyperthreading? I didn't notice that problem during sieving.

 2021-09-02, 21:28 #58 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22×1,249 Posts I leave all CADO processes the default two-threaded unless I run low on memory. But number of threads per process is separate from number of total threads used- CADO defaults to the total number of hyperthreads on a machine, and that is clearly faster than number of cores. In fact, someone 'round these parts said they run 40-threaded on an 18 core HT machine because the overhead threads for DB recording etc are quite slow. Can't say I've tried that extreme myself... Anyway, if you run 20 threads on a 10-core the specifics don't seem to matter. Last fiddled with by VBCurtis on 2021-09-02 at 21:28
 2021-09-07, 12:06 #59 bur     Aug 2020 79*6581e-4;3*2539e-3 3·7·19 Posts Ok, good to know. Does it apply to sieving as well? I never observed non-utilized cores during sieving. I started sieving a c167 from 1992:1644 on Sunday using a polynomial I searched previously with the parameters from your file but instead using admax=5e6 instead of 1e6. Cownoise is 5.7e-13 so nothing too extreme, but will this interfere with adjusting the other parameters? Specifically in my two runs on ~165 digit numbers I had a low number of duplicates (+70% uniques) and could go with 190M rels. Though I think that the quality of the poly does not influence the number of duplicates? Unfortunately I never checked the polys from those two previous composites. Last fiddled with by bur on 2021-09-07 at 12:07
 2021-09-07, 14:22 #60 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22·1,249 Posts More poly select doesn't interfere with anything else- perhaps 2x more poly select saves you 5% of sieve time; that's around half a digit of difficulty. I have poly select calibrated to take about 5% of the total job time; if you double poly select and find a poly that sieves 5% faster, the net gain in job time is roughly zero. The number of uniques varies from poly to poly, and not in a way correlated to score. A much-better poly might sieve over a substantially smaller Q-region, which ought to lead to fewer duplicates, but otherwise it's pretty random.
 2021-09-08, 06:04 #61 bur     Aug 2020 79*6581e-4;3*2539e-3 1100011112 Posts In an older post I saw why that initial c163 sieved so fast: Code: skew 8385289.96, size 8.965e-16, alpha -7.745, combined = 1.077e-12 rroots = 5 The record is 1.096e-12. Last fiddled with by bur on 2021-09-08 at 06:05
 2021-09-19, 16:08 #62 bur     Aug 2020 79*6581e-4;3*2539e-3 18F16 Posts I'm currently at mksol for the c167 (took 197M relations). I started sieving at q = 17M. Since you mentioned earlier that a value of 17M was likely too large and the unique percentage is high (76%) I wanted to see what happens when I resieve the lower q-ranges and try and build a matrix with those included. Could you post the appropriate command line for las? I checked the help but there are a lot of parameters that apparently aren't used normally in the command line, at least from what I remember seeing in top.
2021-09-20, 01:23   #63
swellman

Jun 2012

3,203 Posts

Quote:
 Originally Posted by bur I'm currently at mksol for the c167 (took 197M relations). I started sieving at q = 17M. Since you mentioned earlier that a value of 17M was likely too large and the unique percentage is high (76%) I wanted to see what happens when I resieve the lower q-ranges and try and build a matrix with those included. Could you post the appropriate command line for las? I checked the help but there are a lot of parameters that apparently aren't used normally in the command line, at least from what I remember seeing in top.
See https://www.mersenneforum.org/showthread.php?t=26319 for a worked example using las. Might not have all the answers you seek but it could help.

 2021-09-20, 09:04 #64 bur     Aug 2020 79*6581e-4;3*2539e-3 3·7·19 Posts Thanks, that's very helpful. I also found that the commandline is saved to the sieving file.
 2021-09-23, 16:00 #65 bur     Aug 2020 79*6581e-4;3*2539e-3 3×7×19 Posts I did some tests on a C167 to see the differences between various q-min values. I sieved in the region q0=10M to q1=108M with the parameters from curtis' file. The I concatenated the sieve files of various regions and made msieve build a matrix. These are the results: Code: -- q=15M-104M (89M) 202836566 relations 50758954 duplicates and 152199568 unique relations matrix is 10451195 x 10451420 (3780.8 MB) with weight 983955554 (94.15/col) sparse part has weight 886591531 (84.83/col) -- -- q=15M-105M (90M) 204611251 relations 51190337 duplicates and 153542870 unique relations matrix is 10205132 x 10205356 (3689.9 MB) with weight 960718330 (94.14/col) sparse part has weight 865223080 (84.78/col) -- -- q=15M-106M (91M) 206510998 relations 51623746 duplicates and 154887252 unique relations matrix is 9999089 x 9999314 (3614.4 MB) with weight 941369223 (94.14/col) sparse part has weight 847506249 (84.76/col) -- -- q=16M-105M (89M) 201149205 relations found 49063275 duplicates and 152207886 unique relations matrix is 10458080 x 10458305 (3784.1 MB) with weight 984860664 (94.17/col) sparse part has weight 887403885 (84.85/col) -- -- q=16M-106M (90M) 202926996 relations found 49492280 duplicates and 153556672 unique relations matrix is 10212921 x 10213146 (3693.3 MB) with weight 961603325 (94.15/col) sparse part has weight 866042288 (84.80/col) -- -- q=16M-107M (91M) 204688978 relations 49920242 duplicates and 154890692 unique relations matrix is 10010922 x 10011147 (3618.0 MB) with weight 942278762 (94.12/col) sparse part has weight 848335435 (84.74/col) -- -- q=17M-106M (89M) 199442289 relations found 47446144 duplicates and 152118101 unique relations matrix is 10476553 x 10476778 (3789.4 MB) with weight 986164792 (94.13/col) sparse part has weight 888588171 (84.82/col) -- -- q=17M-107M (90M) 201204271 relations found 47869683 duplicates and 153456544 unique relations matrix is 10231547 x 10231772 (3699.3 MB) with weight 963128811 (94.13/col) sparse part has weight 867441644 (84.78/col) -- -- q=17M-108M (91M) 202962678 relations 48294306 duplicates and 154790328 unique relations matrix is 10028735 x 10028960 (3624.6 MB) with weight 943951416 (94.12/col) sparse part has weight 849864687 (84.74/col) -- I compared runs with the same length of the q-range, i.e. 89M, 90M and 91M. Strangely a q-min of 16M is the best choice in terms of uniques yield, while the smallest matrix was achieved with q-min of 15M. I don't know exactly how many unique relations are required to build a matrix, but 150.7M failed, 152M succeeded. My conclusion would be to start at q0=15M with rels_wanted of 203M-205M. Are there any more ranges or other tests that would be of interest?
 2021-09-23, 16:40 #66 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22×1,249 Posts I'd like to see 10-100M and 10-105M. The CADO group reports that duplicate ratio rises meaningfully when q-max is beyond 8 * q-min. Your data is all within that ratio, but you started sieving at 10M so you have a chance to measure a range outside that ratio to see how duplicates and matrix building behave. A faster test would be to try remdups on 15-108, 12-108. 10-108; we can do some subtraction to see how the duplicate ratio is on specifically 12-15 and 10-12 within the data set of "sieved to 108M". Those Q-ranges look much faster than higher ranges, but if there are over 50% duplicates down there (when filtered with the entire dataset) then the faster sieving is an illusion. We don't need full filtering / matrix generation runs there, just the part until "xxx raw relations, yyy unique".

 Similar Threads Thread Thread Starter Forum Replies Last Post EdH CADO-NFS 127 2020-10-07 01:47 storm5510 Information & Answers 4 2019-11-30 21:32 ksteczk PrimeNet 6 2018-03-26 15:11 R.D. Silverman Cunningham Tables 14 2010-09-29 19:56 R.D. Silverman Cunningham Tables 11 2006-03-06 18:46

All times are UTC. The time now is 11:33.

Thu Oct 21 11:33:24 UTC 2021 up 90 days, 6:02, 1 user, load averages: 1.35, 1.32, 1.29

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.