20190411, 21:54  #342 
"Ed Hall"
Dec 2009
Adirondack Mtns
5×11×61 Posts 
Some Disconcerting ETAs, but then "All Is Well..."
Code:
Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 17:20:51 2019 [0.143 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 18:10:45 2019 [0.285 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 19:00:35 2019 [0.428 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 19:50:28 2019 [0.570 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 20:40:24 2019 [0.713 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 21:30:19 2019 [0.855 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 22:20:10 2019 [0.998 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 23:10:04 2019 [1.140 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Thu Apr 11 23:59:56 2019 [1.283 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 00:49:46 2019 [1.425 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 01:39:38 2019 [1.568 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 02:29:32 2019 [1.710 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 03:19:24 2019 [1.853 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 04:09:15 2019 [1.995 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 04:59:07 2019 [2.138 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 05:48:58 2019 [2.280 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 06:38:50 2019 [2.423 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 07:28:45 2019 [2.565 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 08:18:38 2019 [2.708 s/iter] Info:Linear Algebra: mksol: N=1000 ; ETA (N=21000): Fri Apr 12 09:08:29 2019 [2.850 s/iter] Info:Linear Algebra: mksol: N=860 ; ETA (N=21000): Fri Apr 12 12:40:41 2019 [3.456 s/iter] Info:Quadratic Characters: Starting Info:Square Root: Starting Info:Square Root: Creating file of (a,b) values Info:Square Root: finished Info:Square Root: Factors: 76011547790026822726326568236942510573210375997181897566667718127 635610967526542663809596695842346695879983625491514026964044776163 Info:Polynomial Selection (size optimized): Aggregate statistics: Info:Polynomial Selection (size optimized): potential collisions: 38210.8 Info:Polynomial Selection (size optimized): raw lognorm (nr/min/av/max/std): 38840/38.730/46.683/50.720/0.852 Info:Polynomial Selection (size optimized): optimized lognorm (nr/min/av/max/std): 38840/37.740/42.075/47.510/1.204 Info:Polynomial Selection (size optimized): Total time: 7618.67 Info:Polynomial Selection (root optimized): Aggregate statistics: Info:Polynomial Selection (root optimized): Total time: 2994.82 Info:Polynomial Selection (root optimized): Rootsieve time: 2993.53 Info:Generate Factor Base: Total cpu/real time for makefb: 50.58/11.3882 Info:Generate Free Relations: Total cpu/real time for freerel: 311.62/45.3888 Info:Lattice Sieving: Aggregate statistics: Info:Lattice Sieving: Total number of relations: 27681208 Info:Lattice Sieving: Average J: 7707.97 for 87893 specialq, max bucket fill bkmult 1.0,1s:1.113920 Info:Lattice Sieving: Total time: 181352s Info:Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 73.17/198.683 Info:Filtering  Duplicate Removal, splitting pass: Aggregate statistics: Info:Filtering  Duplicate Removal, splitting pass: CPU time for dup1: 198.3s Info:Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 424.32/391.023 Info:Filtering  Duplicate Removal, removal pass: Aggregate statistics: Info:Filtering  Duplicate Removal, removal pass: CPU time for dup2: 367.7s Info:Filtering  Singleton removal: Total cpu/real time for purge: 368.81/174.105 Info:Filtering  Merging: Total cpu/real time for merge: 879.66/789.756 Info:Filtering  Merging: Total cpu/real time for replay: 67.5/62.5504 Info:Linear Algebra: Total cpu/real time for bwc: 32922.6/8766.06 Info:Linear Algebra: Aggregate statistics: Info:Linear Algebra: Krylov: WCT time 5463.93, iteration CPU time 0.12, COMM 0.01, cpuwait 0.0, commwait 0.0 (42000 iterations) Info:Linear Algebra: Lingen CPU time 360.93, WCT time 104.03 Info:Linear Algebra: Mksol: WCT time 2974.44, iteration CPU time 0.13, COMM 0.01, cpuwait 0.0, commwait 0.0 (21000 iterations) Info:Quadratic Characters: Total cpu/real time for characters: 59.92/19.9351 Info:Square Root: Total cpu/real time for sqrt: 3066.22/474.952 Info:HTTP server: Shutting down HTTP server Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 418847/14039.8 76011547790026822726326568236942510573210375997181897566667718127 635610967526542663809596695842346695879983625491514026964044776163 nfsDone 100% 276 0.3KB/s 00:00 user@computer:~$ date Thu Apr 11 17:28:48 EDT 2019 
20190411, 23:34  #343 
"Curtis"
Feb 2005
Riverside, CA
1000011100101_{2} Posts 
All of my params files include a target number of relations, so I would be able to tell when your runs use one of my files because the number of relations found would be just over my target number set.
Also, my files use a substantially larger LP bound than default, so the number of relations is quite a bit larger than default settings. The only item I'd want that wasn't in your paste of the data dump is the size of the input number. For my own datagathering, I record Qrange sieved (which isn't in the summary, but also isn't that important), poly select time (size and root, but total is enough), sieve time, bwc time, wallclock time. Recently I've been also recording the total weight of the matrix (the number printed just before linear algebra starts), so that I could see how various density selections influence matrix size; again, not too important but if we start collaborating on C150+ settings it would be nice to have. 
20190412, 03:12  #344 
"Ed Hall"
Dec 2009
Adirondack Mtns
110100011011_{2} Posts 
I encoded the composite size in "<XXX>" right after the N in the first line:
Code:
N<138> = 107858998268122985412779892463164903278148452826296404480200645178455386381787979632919305998206531705192000227253731053070278282139941833 I'll work on adding your list items. Should I trim the rest, or leave everything. Actually, I'm wondering if you were referring to the correct post. I had two posts earlier. The first was the samples for a C138, C133 and C135. The second was to point out a discrepancy in the LA mksol ETAs. Do any of the interested parties watch this thread for this type of info, or should I try to post to the CADONFS project? 
20190412, 03:54  #345 
"Curtis"
Feb 2005
Riverside, CA
5^{2}×173 Posts 
I see the composite size now, that's an excellent way to encode it.
I don't know what is valuable for other people; the timings in the summary are all that's really needed to evaluate the params for a composite. 
20190412, 19:10  #346 
"Ed Hall"
Dec 2009
Adirondack Mtns
110100011011_{2} Posts 
I realize these have duplication, but I haven't figured out how to automatically remove duplicate lines, yet.
Anyway, Is all the info you need contained in the following three samples? C86, C129, C121: Code:
N<86> = 56667759878435851136396148508097449099773621318662989162599675933873393187564668667973 Lattice Sieving: params = {'maxresubmit': 5, 'lim0': 393010, 'workdir': '/tmp/cadofactor', 'maxwuerror': 2, 'name': 'c85', 'rels_wanted': 0, 'wutimeout': 10800, 'lim1': 551399, 'qmin': 146453, 'sqside': 1, 'run': True, 'gzip': True, 'maxtimedout': 100, 'qrange': 1000, 'maxwu': 10, 'maxfailed': 100} Polynomial Selection (root optimized): Finished, best polynomial from file /tmp/cadofactor/c85.upload/c85.polyselect2.dq2cf04y.opt_12 has Murphy_E = 2.95e06 Generate Factor Base: Total cpu/real time for makefb: 0.47/0.135556 Generate Free Relations: Total cpu/real time for freerel: 3.85/0.515574 Lattice Sieving: sieving.run(): Task state: {'wu_submitted': 0, 'wu_timedout': 0, 'start_achievement': 1, 'qnext': 146453, 'start_real_time': 0, 'rels_wanted': 440080, 'wu_received': 0, 'wu_failed': 0, 'rels_found': 0} Lattice Sieving: Total number of relations: 470449 Lattice Sieving: Total time: 518.71s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 1.06/0.725687 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 3.44/1.06687 Filtering  Singleton removal: Total cpu/real time for purge: 0.6/0.373629 Lattice Sieving: sieving.run(): Task state: {'wu_submitted': 219, 'rels_wanted': 498934, 'start_achievement': 0.004069714597345937, 'rels_found': 470449, 'qnext': 365000, 'wu_failed': 0, 'stats_avg_J': '1002.231756149383 12237', 'wu_timedout': 0, 'stats_total_cpu_time': '912.3099999999998', 'start_real_time': 3763830410.102031, 'wu_received': 153, 'stats_total_time': '518.71', 'stats_max_bucket_fill': '1.0,1s:1.594150'} Lattice Sieving: sieving.run(): Task state: {'wu_submitted': 285, 'rels_wanted': 522895, 'start_achievement': 0.004069714597345937, 'rels_found': 504252, 'qnext': 431000, 'wu_failed': 0, 'stats_avg_J': '1001.8836890243903 13120', 'wu_timedout': 0, 'stats_total_cpu_time': '972.5799999999998', 'start_real_time': 3763830410.102031, 'wu_received': 164, 'stats_total_time': '551.3100000000002', 'stats_max_bucket_fill': '1.0,1s:1.594150'} Filtering  Merging: Merged matrix has 41245 rows and total weight 7011876 (170.0 entries per row on average) Square Root: Factors: 358031419880876938192954600148936974177 158275940969901931610142052723104530012975662949 Generate Factor Base: Total cpu/real time for makefb: 0.47/0.135556 Generate Free Relations: Total cpu/real time for freerel: 3.85/0.515574 Lattice Sieving: Total number of relations: 651829 Lattice Sieving: Total time: 722.52s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 1.46/1.11501 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 5.82/2.44343 Filtering  Singleton removal: Total cpu/real time for purge: 5.91/2.61454 Filtering  Merging: Total cpu/real time for merge: 13.92/9.40176 Filtering  Merging: Total cpu/real time for replay: 1.34/1.11267 Linear Algebra: Total cpu/real time for bwc: 27.41/8.48 Quadratic Characters: Total cpu/real time for characters: 1.43/0.41662 Square Root: Total cpu/real time for sqrt: 31.34/4.62864 Complete Factorization: Total cpu/elapsed time for entire factorization: 3518.26/98.8682 ==================================================== N<129> = 148184180288647082302426937158351493009802210451497766956818199852902417872086615646923058242179971571945498904372691387599121333 Lattice Sieving: params = {'lim1': 44217255, 'qmin': 711270, 'maxresubmit': 5, 'maxwu': 10, 'rels_wanted': 0, 'run': True, 'maxwuerror': 2, 'qrange': 10000, 'wutimeout': 10800, 'sqside': 1, 'name': 'c130', 'workdir': '/tmp/cadofactor', 'gzip': True, 'lim0': 13124945, 'maxtimedout': 100, 'maxfailed': 100} Polynomial Selection (root optimized): Finished, best polynomial from file /tmp/cadofactor/c130.upload/c130.polyselect2.qxg0sxtu.opt_48 has Murphy_E = 5.93e07 Generate Factor Base: Total cpu/real time for makefb: 50/10.6005 Generate Free Relations: Total cpu/real time for freerel: 311.55/40.6101 Lattice Sieving: sieving.run(): Task state: {'qnext': 711270, 'rels_found': 0, 'start_achievement': 1, 'wu_submitted': 0, 'wu_received': 0, 'wu_timedout': 0, 'rels_wanted': 22129742, 'start_real_time': 0, 'wu_failed': 0} Lattice Sieving: Total number of relations: 22630418 Lattice Sieving: Total time: 101935s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 59.73/158.776 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 312.5/289.52 Filtering  Singleton removal: Total cpu/real time for purge: 64.6/62.8826 Lattice Sieving: sieving.run(): Task state: {'stats_avg_J': '7660.828321977037 47991', 'qnext': 2220000, 'wu_submitted': 151, 'stats_total_time': '101935.03999999998', 'start_real_time': 3763831072.010403, 'rels_wanted': 22856722, 'start_achievement': 0.013695957232578672, 'rels_found': 22630418, 'stats_max_bucket_fill': '1.0,1s:1.120020', 'wu_timedout': 0, 'stats_total_cpu_time': '195411.35000000003', 'wu_received': 67, 'wu_failed': 0} Filtering  Merging: Merged matrix has 1228821 rows and total weight 208900009 (170.0 entries per row on average) Square Root: Factors: 12674601840486459448841848682136209646192693900874822145430131644978522484248447 11691426851398408680908975753372972164838725377739 Generate Factor Base: Total cpu/real time for makefb: 50/10.6005 Generate Free Relations: Total cpu/real time for freerel: 311.55/40.6101 Lattice Sieving: Total number of relations: 25564242 Lattice Sieving: Total time: 120205s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 67.39/179.278 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 396.05/371.704 Filtering  Singleton removal: Total cpu/real time for purge: 337.35/309.169 Filtering  Merging: Total cpu/real time for merge: 750.5/700.049 Filtering  Merging: Total cpu/real time for replay: 47.22/38.9662 Linear Algebra: Total cpu/real time for bwc: 24120.6/6204.46 Quadratic Characters: Total cpu/real time for characters: 56.22/15.795 Square Root: Total cpu/real time for sqrt: 3060.65/418.769 Complete Factorization: Total cpu/elapsed time for entire factorization: 304850/10683 ==================================================== N<121> = 8752900354798992135535713351999727922615660863269877993396052619685931654568649609816781542442396746201949471440217022437 Lattice Sieving: params = {'lim0': 3000000, 'gzip': True, 'maxwuerror': 2, 'maxtimedout': 100, 'run': True, 'qmin': 1000000, 'qrange': 10000, 'rels_wanted': 0, 'maxresubmit': 5, 'wutimeout': 10800, 'workdir': '/tmp/cadofactor', 'sqside': 1, 'maxfailed': 100, 'maxwu': 10, 'lim1': 5500000, 'name': 'c120'} Polynomial Selection (root optimized): Finished, best polynomial from file /tmp/cadofactor/c120.upload/c120.polyselect2.v5ylhooc.opt_90 has Murphy_E = 1.32e06 Generate Factor Base: Total cpu/real time for makefb: 6.22/1.45258 Generate Free Relations: Total cpu/real time for freerel: 155.53/20.4213 Lattice Sieving: sieving.run(): Task state: {'start_real_time': 0, 'start_achievement': 1, 'wu_submitted': 0, 'rels_found': 0, 'wu_timedout': 0, 'wu_failed': 0, 'wu_received': 0, 'rels_wanted': 11474681, 'qnext': 1000000} Lattice Sieving: Total number of relations: 11499970 Lattice Sieving: Total time: 42836.9s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 29.66/71.9494 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 125.83/53.849 Filtering  Singleton removal: Total cpu/real time for purge: 21.02/9.09563 Lattice Sieving: sieving.run(): Task state: {'start_achievement': 0.003286453017735308, 'wu_submitted': 453, 'stats_max_bucket_fill': '1.0,1s:1.360290', 'wu_received': 370, 'rels_wanted': 11964241, 'stats_total_time': '42836.850000000006', 'start_real_time': 3763877574.831329, 'stats_avg_J': '1913.329901275031 250190', 'qnext': 5530000, 'stats_total_cpu_time': '79603.62000000001', 'rels_found': 11499970, 'wu_timedout': 0, 'wu_failed': 0} Lattice Sieving: sieving.run(): Task state: {'start_achievement': 0.003286453017735308, 'wu_submitted': 464, 'stats_max_bucket_fill': '1.0,1s:1.360290', 'wu_received': 429, 'rels_wanted': 13184371, 'stats_total_time': '49254.79', 'start_real_time': 3763877574.831329, 'stats_avg_J': '1911.9272845857893 288247', 'qnext': 5640000, 'stats_total_cpu_time': '91392.13000000002', 'rels_found': 12923469, 'wu_timedout': 0, 'wu_failed': 0} Lattice Sieving: sieving.run(): Task state: {'start_achievement': 0.003286453017735308, 'wu_submitted': 475, 'stats_max_bucket_fill': '1.0,1s:1.360290', 'wu_received': 446, 'rels_wanted': 13482396, 'stats_total_time': '51120.21000000001', 'start_real_time': 3763877574.831329, 'stats_avg_J': '1911.4947376864536 299203', 'qnext': 5750000, 'stats_total_cpu_time': '94879.38000000002', 'rels_found': 13325503, 'wu_timedout': 0, 'wu_failed': 0} Lattice Sieving: sieving.run(): Task state: {'start_achievement': 0.003286453017735308, 'wu_submitted': 536, 'stats_max_bucket_fill': '1.0,1s:1.360290', 'wu_received': 453, 'rels_wanted': 13611673, 'stats_total_time': '51708.15000000001', 'start_real_time': 3763877574.831329, 'stats_avg_J': '1911.302068579334 303590', 'qnext': 6360000, 'stats_total_cpu_time': '95994.38000000002', 'rels_found': 13484536, 'wu_timedout': 0, 'wu_failed': 0} Filtering  Merging: Merged matrix has 742731 rows and total weight 74273389 (100.0 entries per row on average) Square Root: Factors: 63177231098564251400862698002146040730555401588791894828480702706907 138545172091245831321071555847351711191164740172937791 Generate Factor Base: Total cpu/real time for makefb: 6.22/1.45258 Generate Free Relations: Total cpu/real time for freerel: 155.53/20.4213 Lattice Sieving: Total number of relations: 13906880 Lattice Sieving: Total time: 53405.4s Filtering  Duplicate Removal, splitting pass: Total cpu/real time for dup1: 35.98/78.1032 Filtering  Duplicate Removal, removal pass: Total cpu/real time for dup2: 226.18/131.068 Filtering  Singleton removal: Total cpu/real time for purge: 160.12/80.2486 Filtering  Merging: Total cpu/real time for merge: 169.38/142.37 Filtering  Merging: Total cpu/real time for replay: 19.23/14.9333 Linear Algebra: Total cpu/real time for bwc: 5288.67/1377.31 Quadratic Characters: Total cpu/real time for characters: 28.14/7.54837 Square Root: Total cpu/real time for sqrt: 1244.19/172.8 Complete Factorization: Total cpu/elapsed time for entire factorization: 141585/3000.22 
20190412, 22:20  #347 
"Curtis"
Feb 2005
Riverside, CA
5^{2}·173 Posts 
Sure, looks like enough to make some observations.
On your C129, sieve time was 120ksec, while bwc was 24ksec; in wall clock time, bwc was 6200 of your 10600 sec job. On your C121, sieve time 53ksec, bwc 5300 sec; in wall clock time, bwc was 1300 of your 3000 sec job. So, the C121 was oversieved enough to make bwc time 1/10th of sieve time, resulting in a substantial savings in wall clock time. The files I posted target bwc time to be 1/6th to 1/5th of sieving time; I think your C121 shows the way for your setup at bwc time = 1/10th of sieving time. Factoring a C120 in under an hour is pretty snazzy. 
20190413, 03:18  #348  
"Ed Hall"
Dec 2009
Adirondack Mtns
5·11·61 Posts 
Quote:
Anyway, here is a file of data for 87 runs ranging from 74 through 138 dd. It does include all previous samples. These are with unmodified params files from the original developmental version from a few days ago: Last fiddled with by EdH on 20190413 at 03:19 

20190413, 03:57  #349 
"Curtis"
Feb 2005
Riverside, CA
5^{2}·173 Posts 
I'm running a "last" test of new C120 and C130 files; if results fit the timevssize curve I expect, I'll have them posted in the morning for you to the params thread. C125 is still a little slower than the curve suggests should be possible, I still have some tweaking to try there to get a bit more speed.

20190413, 22:58  #350 
"Ed Hall"
Dec 2009
Adirondack Mtns
5·11·61 Posts 
The Last Four Unmodified Runs
I have installed all your modified params files from the other thread. These are the last four unmodified that ran. Of course, I realize that not all of the params files in my current set are modified, but I'm considering the next sets as Modified:

20190415, 03:41  #351 
"Ed Hall"
Dec 2009
Adirondack Mtns
3355_{10} Posts 
Of All the STUPID Things I've Done!!
I accidentally wiped out all my scripts that ran my Aliquot Sequences on my controlling machine  the ones that communicated with the db and ran the ecm and nfs scripts!
The nfs script itself was untouched and I've found an earlier backup for the main controlling script, which is quite fortunate, since I remember having to solve several issues and I have no recollection of how I did solve them. I would have had to research and relearn a bunch. Still, reconstructing all the other scripts may be time consuming, so it might be a while before I get any more data compiled and uploaded. BTW, in case you're interested, I used a recovery program to try to bring them back, but was only able to get the filenames, which unfortunately overwrote the original "deleted" files with zerobyte replacements. 
20190418, 03:18  #352 
"Ed Hall"
Dec 2009
Adirondack Mtns
5×11×61 Posts 
Back Factoring...
I have the scripts pretty much rewritten and they are "mostly" working correctly.
Here is a file of Info for 91 composites factored using VBCurtis' params files, where available. (There may be duplicate runs for some of the numbers.) 
Thread Tools  
Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
CADONFS on windows  jux  CADONFS  22  20191112 12:08 
CADO help  henryzz  CADONFS  4  20171120 15:14 
CADO and WinBlows  akruppa  Programming  22  20151231 08:37 
CADONFS  skan  Information & Answers  1  20131022 07:00 
CADO  R.D. Silverman  Factoring  4  20081106 12:35 