Msieve failing

pinhodecarlos · 2018-11-08, 22:09

Jon's Msieve 988 run for 6_323plus5_323, td=130, no limit on relations.

Code:

Thu Nov  8 08:06:34 2018  skipped 120 relations with b > 2^32
Thu Nov  8 08:06:34 2018  skipped 1140 relations with composite factors
Thu Nov  8 08:06:34 2018  found 56656301 hash collisions in 252848582 relations
Thu Nov  8 08:06:45 2018  added 1219718 free relations
Thu Nov  8 08:06:45 2018  commencing duplicate removal, pass 2
Thu Nov  8 08:10:10 2018  found 58796653 duplicates and 195271647 unique relations
Thu Nov  8 08:10:10 2018  memory use: 1449.5 MB
Thu Nov  8 08:10:10 2018  reading ideals above 720000
Thu Nov  8 08:10:10 2018  commencing singleton removal, initial pass
Thu Nov  8 08:30:17 2018  memory use: 5512.0 MB
Thu Nov  8 08:30:18 2018  reading all ideals from disk
Thu Nov  8 08:30:21 2018  memory use: 7758.5 MB
Thu Nov  8 08:30:38 2018  keeping 176343462 ideals with weight <= 200, target excess is 1006406
Thu Nov  8 08:30:58 2018  commencing in-memory singleton removal
Thu Nov  8 08:31:12 2018  begin with 195271647 relations and 176343462 unique ideals
Thu Nov  8 08:33:39 2018  reduce to 128966320 relations and 103483492 ideals in 13 passes
Thu Nov  8 08:33:39 2018  max relations containing the same ideal: 158
Thu Nov  8 08:34:29 2018  removing 10720772 relations and 8720772 ideals in 2000000 cliques
Thu Nov  8 08:34:33 2018  commencing in-memory singleton removal
Thu Nov  8 08:34:41 2018  begin with 118245548 relations and 103483492 unique ideals
Thu Nov  8 08:35:58 2018  reduce to 117609364 relations and 94114298 ideals in 8 passes
Thu Nov  8 08:35:58 2018  max relations containing the same ideal: 150
Thu Nov  8 08:36:44 2018  removing 8144978 relations and 6144978 ideals in 2000000 cliques
Thu Nov  8 08:36:46 2018  commencing in-memory singleton removal
Thu Nov  8 08:36:54 2018  begin with 109464386 relations and 94114298 unique ideals
Thu Nov  8 08:37:56 2018  reduce to 109041539 relations and 87539319 ideals in 7 passes
Thu Nov  8 08:37:56 2018  max relations containing the same ideal: 142
Thu Nov  8 08:38:38 2018  removing 7366219 relations and 5366219 ideals in 2000000 cliques
Thu Nov  8 08:38:41 2018  commencing in-memory singleton removal
Thu Nov  8 08:38:48 2018  begin with 101675320 relations and 87539319 unique ideals
Thu Nov  8 08:39:45 2018  reduce to 101307614 relations and 81799083 ideals in 7 passes
Thu Nov  8 08:39:45 2018  max relations containing the same ideal: 136
Thu Nov  8 08:40:24 2018  removing 6958609 relations and 4958609 ideals in 2000000 cliques
Thu Nov  8 08:40:27 2018  commencing in-memory singleton removal
Thu Nov  8 08:40:33 2018  begin with 94349005 relations and 81799083 unique ideals
Thu Nov  8 08:41:34 2018  reduce to 94007359 relations and 76492739 ideals in 8 passes
Thu Nov  8 08:41:34 2018  max relations containing the same ideal: 128
Thu Nov  8 08:42:10 2018  removing 6699281 relations and 4699281 ideals in 2000000 cliques
Thu Nov  8 08:42:13 2018  commencing in-memory singleton removal
Thu Nov  8 08:42:18 2018  begin with 87308078 relations and 76492739 unique ideals
Thu Nov  8 08:43:07 2018  reduce to 86977608 relations and 71456798 ideals in 7 passes
Thu Nov  8 08:43:07 2018  max relations containing the same ideal: 125
Thu Nov  8 08:43:41 2018  removing 6513452 relations and 4513452 ideals in 2000000 cliques
Thu Nov  8 08:43:44 2018  commencing in-memory singleton removal
Thu Nov  8 08:43:49 2018  begin with 80464156 relations and 71456798 unique ideals
Thu Nov  8 08:44:33 2018  reduce to 80136415 relations and 66609157 ideals in 7 passes
Thu Nov  8 08:44:33 2018  max relations containing the same ideal: 116
Thu Nov  8 08:45:05 2018  removing 6375808 relations and 4375808 ideals in 2000000 cliques
Thu Nov  8 08:45:07 2018  commencing in-memory singleton removal
Thu Nov  8 08:45:12 2018  begin with 73760607 relations and 66609157 unique ideals
Thu Nov  8 08:45:52 2018  reduce to 73431085 relations and 61896821 ideals in 7 passes
Thu Nov  8 08:45:52 2018  max relations containing the same ideal: 110
Thu Nov  8 08:46:21 2018  removing 6268705 relations and 4268705 ideals in 2000000 cliques
Thu Nov  8 08:46:23 2018  commencing in-memory singleton removal
Thu Nov  8 08:46:28 2018  begin with 67162380 relations and 61896821 unique ideals
Thu Nov  8 08:47:04 2018  reduce to 66820234 relations and 57278058 ideals in 7 passes
Thu Nov  8 08:47:04 2018  max relations containing the same ideal: 103
Thu Nov  8 08:47:31 2018  removing 6189575 relations and 4189575 ideals in 2000000 cliques
Thu Nov  8 08:47:33 2018  commencing in-memory singleton removal
Thu Nov  8 08:47:37 2018  begin with 60630659 relations and 57278058 unique ideals
Thu Nov  8 08:48:09 2018  reduce to 60268770 relations and 52717485 ideals in 7 passes
Thu Nov  8 08:48:09 2018  max relations containing the same ideal: 95
Thu Nov  8 08:48:33 2018  removing 6140256 relations and 4140256 ideals in 2000000 cliques
Thu Nov  8 08:48:35 2018  commencing in-memory singleton removal
Thu Nov  8 08:48:38 2018  begin with 54128514 relations and 52717485 unique ideals
Thu Nov  8 08:49:11 2018  reduce to 53733929 relations and 48171354 ideals in 8 passes
Thu Nov  8 08:49:11 2018  max relations containing the same ideal: 88
Thu Nov  8 08:49:32 2018  removing 6116168 relations and 4116168 ideals in 2000000 cliques
Thu Nov  8 08:49:34 2018  commencing in-memory singleton removal
Thu Nov  8 08:49:37 2018  begin with 47617761 relations and 48171354 unique ideals
Thu Nov  8 08:50:02 2018  reduce to 47169184 relations and 43592631 ideals in 7 passes
Thu Nov  8 08:50:02 2018  max relations containing the same ideal: 82
Thu Nov  8 08:50:21 2018  removing 6133472 relations and 4133472 ideals in 2000000 cliques
Thu Nov  8 08:50:22 2018  commencing in-memory singleton removal
Thu Nov  8 08:50:25 2018  begin with 41035712 relations and 43592631 unique ideals
Thu Nov  8 08:50:49 2018  reduce to 40495997 relations and 38900200 ideals in 8 passes
Thu Nov  8 08:50:49 2018  max relations containing the same ideal: 76
Thu Nov  8 08:51:05 2018  removing 1763656 relations and 1335290 ideals in 428366 cliques
Thu Nov  8 08:51:06 2018  commencing in-memory singleton removal
Thu Nov  8 08:51:09 2018  begin with 38732341 relations and 38900200 unique ideals
Thu Nov  8 08:51:26 2018  reduce to 38676284 relations and 37508262 ideals in 6 passes
Thu Nov  8 08:51:26 2018  max relations containing the same ideal: 71
Thu Nov  8 08:51:44 2018  relations with 0 large ideals: 38506
Thu Nov  8 08:51:44 2018  relations with 1 large ideals: 24865
Thu Nov  8 08:51:44 2018  relations with 2 large ideals: 311562
Thu Nov  8 08:51:44 2018  relations with 3 large ideals: 1726855
Thu Nov  8 08:51:44 2018  relations with 4 large ideals: 5166141
Thu Nov  8 08:51:44 2018  relations with 5 large ideals: 9197815
Thu Nov  8 08:51:44 2018  relations with 6 large ideals: 10228224
Thu Nov  8 08:51:44 2018  relations with 7+ large ideals: 11982316
Thu Nov  8 08:51:44 2018  commencing 2-way merge

My run, for the same data set, different msieve version: Win 7 64 bits, msieve_SVN1028-trunk_ivy-VBITS128, td=130

Code:

Thu Nov 08 21:00:54 2018  skipped 122 relations with b > 2^32
Thu Nov 08 21:00:54 2018  skipped 1149 relations with composite factors
Thu Nov 08 21:00:54 2018  found 57140517 hash collisions in 254087054 relations
Thu Nov 08 21:01:11 2018  added 1219718 free relations
Thu Nov 08 21:01:11 2018  commencing duplicate removal, pass 2
Thu Nov 08 21:08:17 2018  found 59374049 duplicates and 195932723 unique relations
Thu Nov 08 21:08:17 2018  memory use: 1449.5 MB
Thu Nov 08 21:08:17 2018  reading ideals above 201392128
Thu Nov 08 21:08:17 2018  commencing singleton removal, initial pass
Thu Nov 08 21:33:24 2018  memory use: 3012.0 MB
Thu Nov 08 21:33:24 2018  reading all ideals from disk
Thu Nov 08 21:33:57 2018  memory use: 3262.7 MB
Thu Nov 08 21:34:03 2018  commencing in-memory singleton removal
Thu Nov 08 21:34:09 2018  begin with 195932723 relations and 155220250 unique ideals
Thu Nov 08 21:35:21 2018  reduce to 129903145 relations and 82701537 ideals in 13 passes
Thu Nov 08 21:35:21 2018  max relations containing the same ideal: 34
Thu Nov 08 21:35:30 2018  reading ideals above 720000
Thu Nov 08 21:35:30 2018  commencing singleton removal, initial pass
Thu Nov 08 21:57:31 2018  memory use: 2756.0 MB
Thu Nov 08 21:57:32 2018  reading all ideals from disk
Thu Nov 08 21:57:52 2018  memory use: 5150.9 MB
Thu Nov 08 21:57:54 2018  keeping 58362264 ideals with weight <= 200, target excess is 46644331
Thu Nov 08 21:57:57 2018  commencing in-memory singleton removal
Thu Nov 08 21:57:59 2018  begin with 129903145 relations and 58362264 unique ideals
Thu Nov 08 21:58:03 2018  reduce to 103032554 relations and 0 ideals in 5 passes
Thu Nov 08 21:58:03 2018  max relations containing the same ideal: 0
Thu Nov 08 21:58:04 2018  relations with 0 large ideals: 103032554
Thu Nov 08 21:58:04 2018  relations with 1 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 2 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 3 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 4 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 5 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 6 large ideals: 0
Thu Nov 08 21:58:04 2018  relations with 7+ large ideals: 0

pinhodecarlos · 2018-11-08, 22:19

The above is a 31-bit job. My msieve version has no issue processing 29 and 30-bit jobs.

RichD · 2018-11-08, 22:48

There is an extra eight lines in your log file, which includes another pruning step.

Code:

Thu Nov 08 21:08:17 2018  reading ideals above 201392128
Thu Nov 08 21:08:17 2018  commencing singleton removal, initial pass
Thu Nov 08 21:33:24 2018  memory use: 3012.0 MB
Thu Nov 08 21:33:24 2018  reading all ideals from disk
Thu Nov 08 21:33:57 2018  memory use: 3262.7 MB
Thu Nov 08 21:34:03 2018  commencing in-memory singleton removal
Thu Nov 08 21:34:09 2018  begin with 195932723 relations and 155220250 unique ideals
Thu Nov 08 21:35:21 2018  reduce to 129903145 relations and 82701537 ideals in 13 passes
Thu Nov 08 21:35:21 2018  max relations containing the same ideal: 34

Just a wild guess but is that folder/partition out of disk space?

pinhodecarlos · 2018-11-08, 22:55

I have more than 385GB free on a 500GB drive.

This is a dedicated laptop for NFS, LLR, ecm, etc...though I haven’t updated windows 7 64 bits for more than 5 years. I have the minimum and necessary Windows services to run this forum projects.

VictordeHolland · 2018-11-09, 05:10

Carlos, is the CPU in the laptop, Ivybridge or newer (does it have AVX?)
Did you unzip the relations or did you use the .dat.gz file?
Because 6_323plus5_323 filters just fine here with SVN1028-trunk-Ivy_VBITS128 (I forgot to set TD, but I'll run filtering again with TD=130 to see if that changes anything)

pinhodecarlos · 2018-11-09, 06:12

Victor, PC is a laptop, Ivybridge with AVX. I always unzip the relations file.

VictordeHolland · 2018-11-09, 14:52

SVN1028
TD=120 also works

Code:

weight of 11717699 cycles is about 1406285973 (120.01/cycle)

SVN1028
TD=130 'fails' because not enough cycles

Code:

Fri Nov 09 09:36:37 2018  found 66775 cycles, need 5585123
Fri Nov 09 09:36:37 2018  too few cycles, matrix probably cannot build

I'm not sure why SVN988 does produce a matrix at TD=130 and SVN1028 doesn't, but the difference between TD=130 and TD=120 is small:

SVN988 (Jons)

Code:

weight of 11482785 cycles is about 1493240428 (130.04/cycle)

SVN1028-trunk-ivy_VBITS=128

Code:

weight of 11717699 cycles is about 1406285973 (120.01/cycle)

pinhodecarlos · 2018-11-09, 21:54

Victor, even for td=130 your filtering stage is different than mine.

2018-11-08, 22:55	#4
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 2·3²·281 Posts	I have more than 385GB free on a 500GB drive. This is a dedicated laptop for NFS, LLR, ecm, etc...though I haven’t updated windows 7 64 bits for more than 5 years. I have the minimum and necessary Windows services to run this forum projects. Last fiddled with by pinhodecarlos on 2018-11-08 at 23:00

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
mulmod failing	ATH	Programming	4	2017-06-08 22:22
Aliqueit Failing to Find Msieve Factors	EdH	Msieve	14	2016-09-05 21:52
Worker 1 Keeps failing	zenzu88	Software	2	2012-04-10 15:16
Tried out aliqueit.exe: ggnfs failing	Greebley	Aliquot Sequences	35	2010-02-13 15:23
Failing Torture Test..	jugbugs	Hardware	12	2004-03-25 02:37

2018-11-08, 22:19	#2
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 2×3²×281 Posts	The above is a 31-bit job. My msieve version has no issue processing 29 and 30-bit jobs.

2018-11-09, 06:12	#6
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 11702₈ Posts	Victor, PC is a laptop, Ivybridge with AVX. I always unzip the relations file.

2018-11-09, 21:54	#8
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 11702₈ Posts	Victor, even for td=130 your filtering stage is different than mine.