Msieve failing
 

Jon's Msieve 988 run for 6_323plus5_323, td=130, no limit on relations.


[CODE]
Thu Nov 8 08:06:34 2018 skipped 120 relations with b > 2^32
Thu Nov 8 08:06:34 2018 skipped 1140 relations with composite factors
Thu Nov 8 08:06:34 2018 found 56656301 hash collisions in 252848582 relations
Thu Nov 8 08:06:45 2018 added 1219718 free relations
Thu Nov 8 08:06:45 2018 commencing duplicate removal, pass 2
Thu Nov 8 08:10:10 2018 found 58796653 duplicates and 195271647 unique relations
Thu Nov 8 08:10:10 2018 memory use: 1449.5 MB
[B]Thu Nov 8 08:10:10 2018 reading ideals above 720000[/B]
Thu Nov 8 08:10:10 2018 commencing singleton removal, initial pass
Thu Nov 8 08:30:17 2018 memory use: 5512.0 MB
Thu Nov 8 08:30:18 2018 reading all ideals from disk
Thu Nov 8 08:30:21 2018 memory use: 7758.5 MB
[B]Thu Nov 8 08:30:38 2018 keeping 176343462 ideals with weight <= 200, target excess is 1006406[/B]
Thu Nov 8 08:30:58 2018 commencing in-memory singleton removal
Thu Nov 8 08:31:12 2018 begin with 195271647 relations and 176343462 unique ideals
Thu Nov 8 08:33:39 2018 reduce to 128966320 relations and 103483492 ideals in 13 passes
Thu Nov 8 08:33:39 2018 max relations containing the same ideal: 158
Thu Nov 8 08:34:29 2018 removing 10720772 relations and 8720772 ideals in 2000000 cliques
Thu Nov 8 08:34:33 2018 commencing in-memory singleton removal
Thu Nov 8 08:34:41 2018 begin with 118245548 relations and 103483492 unique ideals
Thu Nov 8 08:35:58 2018 reduce to 117609364 relations and 94114298 ideals in 8 passes
Thu Nov 8 08:35:58 2018 max relations containing the same ideal: 150
Thu Nov 8 08:36:44 2018 removing 8144978 relations and 6144978 ideals in 2000000 cliques
Thu Nov 8 08:36:46 2018 commencing in-memory singleton removal
Thu Nov 8 08:36:54 2018 begin with 109464386 relations and 94114298 unique ideals
Thu Nov 8 08:37:56 2018 reduce to 109041539 relations and 87539319 ideals in 7 passes
Thu Nov 8 08:37:56 2018 max relations containing the same ideal: 142
Thu Nov 8 08:38:38 2018 removing 7366219 relations and 5366219 ideals in 2000000 cliques
Thu Nov 8 08:38:41 2018 commencing in-memory singleton removal
Thu Nov 8 08:38:48 2018 begin with 101675320 relations and 87539319 unique ideals
Thu Nov 8 08:39:45 2018 reduce to 101307614 relations and 81799083 ideals in 7 passes
Thu Nov 8 08:39:45 2018 max relations containing the same ideal: 136
Thu Nov 8 08:40:24 2018 removing 6958609 relations and 4958609 ideals in 2000000 cliques
Thu Nov 8 08:40:27 2018 commencing in-memory singleton removal
Thu Nov 8 08:40:33 2018 begin with 94349005 relations and 81799083 unique ideals
Thu Nov 8 08:41:34 2018 reduce to 94007359 relations and 76492739 ideals in 8 passes
Thu Nov 8 08:41:34 2018 max relations containing the same ideal: 128
Thu Nov 8 08:42:10 2018 removing 6699281 relations and 4699281 ideals in 2000000 cliques
Thu Nov 8 08:42:13 2018 commencing in-memory singleton removal
Thu Nov 8 08:42:18 2018 begin with 87308078 relations and 76492739 unique ideals
Thu Nov 8 08:43:07 2018 reduce to 86977608 relations and 71456798 ideals in 7 passes
Thu Nov 8 08:43:07 2018 max relations containing the same ideal: 125
Thu Nov 8 08:43:41 2018 removing 6513452 relations and 4513452 ideals in 2000000 cliques
Thu Nov 8 08:43:44 2018 commencing in-memory singleton removal
Thu Nov 8 08:43:49 2018 begin with 80464156 relations and 71456798 unique ideals
Thu Nov 8 08:44:33 2018 reduce to 80136415 relations and 66609157 ideals in 7 passes
Thu Nov 8 08:44:33 2018 max relations containing the same ideal: 116
Thu Nov 8 08:45:05 2018 removing 6375808 relations and 4375808 ideals in 2000000 cliques
Thu Nov 8 08:45:07 2018 commencing in-memory singleton removal
Thu Nov 8 08:45:12 2018 begin with 73760607 relations and 66609157 unique ideals
Thu Nov 8 08:45:52 2018 reduce to 73431085 relations and 61896821 ideals in 7 passes
Thu Nov 8 08:45:52 2018 max relations containing the same ideal: 110
Thu Nov 8 08:46:21 2018 removing 6268705 relations and 4268705 ideals in 2000000 cliques
Thu Nov 8 08:46:23 2018 commencing in-memory singleton removal
Thu Nov 8 08:46:28 2018 begin with 67162380 relations and 61896821 unique ideals
Thu Nov 8 08:47:04 2018 reduce to 66820234 relations and 57278058 ideals in 7 passes
Thu Nov 8 08:47:04 2018 max relations containing the same ideal: 103
Thu Nov 8 08:47:31 2018 removing 6189575 relations and 4189575 ideals in 2000000 cliques
Thu Nov 8 08:47:33 2018 commencing in-memory singleton removal
Thu Nov 8 08:47:37 2018 begin with 60630659 relations and 57278058 unique ideals
Thu Nov 8 08:48:09 2018 reduce to 60268770 relations and 52717485 ideals in 7 passes
Thu Nov 8 08:48:09 2018 max relations containing the same ideal: 95
Thu Nov 8 08:48:33 2018 removing 6140256 relations and 4140256 ideals in 2000000 cliques
Thu Nov 8 08:48:35 2018 commencing in-memory singleton removal
Thu Nov 8 08:48:38 2018 begin with 54128514 relations and 52717485 unique ideals
Thu Nov 8 08:49:11 2018 reduce to 53733929 relations and 48171354 ideals in 8 passes
Thu Nov 8 08:49:11 2018 max relations containing the same ideal: 88
Thu Nov 8 08:49:32 2018 removing 6116168 relations and 4116168 ideals in 2000000 cliques
Thu Nov 8 08:49:34 2018 commencing in-memory singleton removal
Thu Nov 8 08:49:37 2018 begin with 47617761 relations and 48171354 unique ideals
Thu Nov 8 08:50:02 2018 reduce to 47169184 relations and 43592631 ideals in 7 passes
Thu Nov 8 08:50:02 2018 max relations containing the same ideal: 82
Thu Nov 8 08:50:21 2018 removing 6133472 relations and 4133472 ideals in 2000000 cliques
Thu Nov 8 08:50:22 2018 commencing in-memory singleton removal
Thu Nov 8 08:50:25 2018 begin with 41035712 relations and 43592631 unique ideals
Thu Nov 8 08:50:49 2018 reduce to 40495997 relations and 38900200 ideals in 8 passes
Thu Nov 8 08:50:49 2018 max relations containing the same ideal: 76
Thu Nov 8 08:51:05 2018 removing 1763656 relations and 1335290 ideals in 428366 cliques
Thu Nov 8 08:51:06 2018 commencing in-memory singleton removal
Thu Nov 8 08:51:09 2018 begin with 38732341 relations and 38900200 unique ideals
Thu Nov 8 08:51:26 2018 reduce to 38676284 relations and 37508262 ideals in 6 passes
Thu Nov 8 08:51:26 2018 max relations containing the same ideal: 71
Thu Nov 8 08:51:44 2018 relations with 0 large ideals: 38506
Thu Nov 8 08:51:44 2018 relations with 1 large ideals: 24865
Thu Nov 8 08:51:44 2018 relations with 2 large ideals: 311562
Thu Nov 8 08:51:44 2018 relations with 3 large ideals: 1726855
Thu Nov 8 08:51:44 2018 relations with 4 large ideals: 5166141
Thu Nov 8 08:51:44 2018 relations with 5 large ideals: 9197815
Thu Nov 8 08:51:44 2018 relations with 6 large ideals: 10228224
Thu Nov 8 08:51:44 2018 relations with 7+ large ideals: 11982316
Thu Nov 8 08:51:44 2018 commencing 2-way merge[/CODE]My run, for the same data set, different msieve version: Win 7 64 bits, msieve_SVN1028-trunk_ivy-VBITS128, td=130


[CODE]
Thu Nov 08 21:00:54 2018 skipped 122 relations with b > 2^32
Thu Nov 08 21:00:54 2018 skipped 1149 relations with composite factors
Thu Nov 08 21:00:54 2018 found 57140517 hash collisions in 254087054 relations
Thu Nov 08 21:01:11 2018 added 1219718 free relations
Thu Nov 08 21:01:11 2018 commencing duplicate removal, pass 2
Thu Nov 08 21:08:17 2018 found 59374049 duplicates and 195932723 unique relations
Thu Nov 08 21:08:17 2018 memory use: 1449.5 MB
[B]Thu Nov 08 21:08:17 2018 reading ideals above 201392128[/B]
Thu Nov 08 21:08:17 2018 commencing singleton removal, initial pass
Thu Nov 08 21:33:24 2018 memory use: 3012.0 MB
Thu Nov 08 21:33:24 2018 reading all ideals from disk
Thu Nov 08 21:33:57 2018 memory use: 3262.7 MB
Thu Nov 08 21:34:03 2018 commencing in-memory singleton removal
Thu Nov 08 21:34:09 2018 begin with 195932723 relations and 155220250 unique ideals
Thu Nov 08 21:35:21 2018 reduce to 129903145 relations and 82701537 ideals in 13 passes
Thu Nov 08 21:35:21 2018 max relations containing the same ideal: 34
Thu Nov 08 21:35:30 2018 reading ideals above 720000
Thu Nov 08 21:35:30 2018 commencing singleton removal, initial pass
Thu Nov 08 21:57:31 2018 memory use: 2756.0 MB
Thu Nov 08 21:57:32 2018 reading all ideals from disk
Thu Nov 08 21:57:52 2018 memory use: 5150.9 MB
[B]Thu Nov 08 21:57:54 2018 keeping 58362264 ideals with weight <= 200, target excess is 46644331[/B]
Thu Nov 08 21:57:57 2018 commencing in-memory singleton removal
Thu Nov 08 21:57:59 2018 begin with 129903145 relations and 58362264 unique ideals
[B]Thu Nov 08 21:58:03 2018 reduce to 103032554 relations and 0 ideals in 5 passes[/B]
Thu Nov 08 21:58:03 2018 max relations containing the same ideal: 0
Thu Nov 08 21:58:04 2018 relations with 0 large ideals: 103032554
Thu Nov 08 21:58:04 2018 relations with 1 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 2 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 3 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 4 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 5 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 6 large ideals: 0
Thu Nov 08 21:58:04 2018 relations with 7+ large ideals: 0[/CODE]

The above is a 31-bit job. My msieve version has no issue processing 29 and 30-bit jobs.

There is an extra eight lines in your log file, which includes another pruning step.
[CODE]Thu Nov 08 21:08:17 2018 reading ideals above 201392128
Thu Nov 08 21:08:17 2018 commencing singleton removal, initial pass
Thu Nov 08 21:33:24 2018 memory use: 3012.0 MB
Thu Nov 08 21:33:24 2018 reading all ideals from disk
Thu Nov 08 21:33:57 2018 memory use: 3262.7 MB
Thu Nov 08 21:34:03 2018 commencing in-memory singleton removal
Thu Nov 08 21:34:09 2018 begin with 195932723 relations and 155220250 unique ideals
Thu Nov 08 21:35:21 2018 reduce to 129903145 relations and 82701537 ideals in 13 passes
Thu Nov 08 21:35:21 2018 max relations containing the same ideal: 34[/CODE]
Just a wild guess but is that folder/partition out of disk space?

I have more than 385GB free on a 500GB drive.

This is a dedicated laptop for NFS, LLR, ecm, etc...though I haven’t updated windows 7 64 bits for more than 5 years. I have the minimum and necessary Windows services to run this forum projects.

6_323plus5_323 filtering works fine here too (SVN1028)
 

Carlos, is the CPU in the laptop, Ivybridge or newer (does it have AVX?)
Did you unzip the relations or did you use the .dat.gz file?
Because 6_323plus5_323 filters just fine here with SVN1028-trunk-Ivy_VBITS128 (I forgot to set TD, but I'll run filtering again with TD=130 to see if that changes anything)

Victor, PC is a laptop, Ivybridge with AVX. I always unzip the relations file.

more filtering data with Msieve SVN1028
 

SVN1028
TD=120 also works
[code]weight of 11717699 cycles is about 1406285973 (120.01/cycle)[/code]SVN1028
TD=130 'fails' because not enough cycles

[code]Fri Nov 09 09:36:37 2018 found 66775 cycles, need 5585123
Fri Nov 09 09:36:37 2018 too few cycles, matrix probably cannot build[/code]I'm not sure why SVN988 does produce a matrix at TD=130 and SVN1028 doesn't, but the difference between TD=130 and TD=120 is small:

SVN988 (Jons)
[code]weight of 11482785 cycles is about 1493240428 (130.04/cycle)[/code] SVN1028-trunk-ivy_VBITS=128[code]weight of 11717699 cycles is about 1406285973 (120.01/cycle)[/code]

Victor, even for td=130 your filtering stage is different than mine.