Linear Algebra error question

sashamkrt · 2013-12-13, 05:42

I started LA for C192 (3,664+1) sieved with LBPA/LBPR 32/32 bit.
And found an error in the log today:
lanczos error: submatrix is not invertible
lanczos halted after 3691 iterations (dim = 233377)
linear algebra failed; retrying...

Is it OK? I wouldn't be nice to wait 2 months and found no solution :)
Or should I sieve more?

Code:

Thu Dec 12 17:43:19 2013  begin with 304618004 relations and 280828360 unique ideals
Thu Dec 12 17:46:11 2013  reduce to 151046622 relations and 106183744 ideals in 19 passes
...
Thu Dec 12 19:01:06 2013  begin with 88316055 relations and 89142839 unique ideals
Thu Dec 12 19:02:29 2013  reduce to 88280680 relations and 87301963 ideals in 7 passes
Thu Dec 12 19:02:29 2013  max relations containing the same ideal: 150
Thu Dec 12 19:03:50 2013  relations with 0 large ideals: 2959
Thu Dec 12 19:03:50 2013  relations with 1 large ideals: 2942
Thu Dec 12 19:03:50 2013  relations with 2 large ideals: 46275
Thu Dec 12 19:03:50 2013  relations with 3 large ideals: 457247
Thu Dec 12 19:03:50 2013  relations with 4 large ideals: 2493916
Thu Dec 12 19:03:50 2013  relations with 5 large ideals: 8263377
Thu Dec 12 19:03:50 2013  relations with 6 large ideals: 17421304
Thu Dec 12 19:03:50 2013  relations with 7+ large ideals: 59592660
Thu Dec 12 19:03:50 2013  commencing 2-way merge
Thu Dec 12 19:05:20 2013  reduce to 54318506 relation sets and 53339789 unique ideals
Thu Dec 12 19:05:20 2013  commencing full merge
Thu Dec 12 19:22:02 2013  memory use: 6468.8 MB
Thu Dec 12 19:22:11 2013  found 30552778 cycles, need 30417989
Thu Dec 12 19:22:14 2013  weight of 30417989 cycles is about 2129542714 (70.01/cycle)
Thu Dec 12 19:22:14 2013  distribution of cycle lengths:
Thu Dec 12 19:22:14 2013  1 relations: 4954457
Thu Dec 12 19:22:14 2013  2 relations: 4352415
Thu Dec 12 19:22:14 2013  3 relations: 4031316
Thu Dec 12 19:22:14 2013  4 relations: 3524105
Thu Dec 12 19:22:14 2013  5 relations: 3060049
Thu Dec 12 19:22:14 2013  6 relations: 2590631
Thu Dec 12 19:22:14 2013  7 relations: 2150918
Thu Dec 12 19:22:14 2013  8 relations: 1728873
Thu Dec 12 19:22:14 2013  9 relations: 1347041
Thu Dec 12 19:22:14 2013  10+ relations: 2678184
Thu Dec 12 19:22:14 2013  heaviest cycle: 18 relations
Thu Dec 12 19:22:22 2013  commencing cycle optimization
Thu Dec 12 19:23:00 2013  start with 142049232 relations
Thu Dec 12 19:26:28 2013  pruned 2649436 relations
Thu Dec 12 19:26:29 2013  memory use: 5060.3 MB
Thu Dec 12 19:26:29 2013  distribution of cycle lengths:
Thu Dec 12 19:26:29 2013  1 relations: 4954457
Thu Dec 12 19:26:29 2013  2 relations: 4441217
Thu Dec 12 19:26:29 2013  3 relations: 4156614
Thu Dec 12 19:26:29 2013  4 relations: 3580726
Thu Dec 12 19:26:29 2013  5 relations: 3104562
Thu Dec 12 19:26:29 2013  6 relations: 2595819
Thu Dec 12 19:26:29 2013  7 relations: 2144587
Thu Dec 12 19:26:29 2013  8 relations: 1705590
Thu Dec 12 19:26:29 2013  9 relations: 1311367
Thu Dec 12 19:26:29 2013  10+ relations: 2423050
Thu Dec 12 19:26:29 2013  heaviest cycle: 18 relations
Thu Dec 12 19:28:28 2013  RelProcTime: 11319
Thu Dec 12 19:28:28 2013  
Thu Dec 12 19:28:28 2013  commencing linear algebra
Thu Dec 12 19:28:34 2013  read 30417989 cycles
Thu Dec 12 19:29:21 2013  cycles contain 87618160 unique relations
Thu Dec 12 19:39:35 2013  read 87618160 relations
Thu Dec 12 19:42:11 2013  using 20 quadratic characters above 4294917296
Thu Dec 12 19:49:33 2013  building initial matrix
Thu Dec 12 20:08:42 2013  memory use: 12433.7 MB
Thu Dec 12 20:10:02 2013  read 30417989 cycles
Thu Dec 12 20:10:15 2013  matrix is 30417811 x 30417989 (9296.2 MB) with weight 2871135383 (94.39/col)
Thu Dec 12 20:10:15 2013  sparse part has weight 2071934493 (68.12/col)
Thu Dec 12 20:16:21 2013  filtering completed in 2 passes
Thu Dec 12 20:16:33 2013  matrix is 30409532 x 30409709 (9295.7 MB) with weight 2870886540 (94.41/col)
Thu Dec 12 20:16:33 2013  sparse part has weight 2071894059 (68.13/col)
Thu Dec 12 20:20:04 2013  matrix starts at (0, 0)
Thu Dec 12 20:20:16 2013  matrix is 30409532 x 30409709 (9295.7 MB) with weight 2870886540 (94.41/col)
Thu Dec 12 20:20:16 2013  sparse part has weight 2071894059 (68.13/col)
Thu Dec 12 20:20:16 2013  saving the first 48 matrix rows for later
Thu Dec 12 20:20:24 2013  matrix includes 64 packed rows
Thu Dec 12 20:20:30 2013  matrix is 30409484 x 30409709 (8970.2 MB) with weight 2300003568 (75.63/col)
Thu Dec 12 20:20:30 2013  sparse part has weight 2047389422 (67.33/col)
Thu Dec 12 20:20:30 2013  using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Thu Dec 12 20:24:43 2013  commencing Lanczos iteration (4 threads)
Thu Dec 12 20:24:43 2013  memory use: 7589.9 MB
Thu Dec 12 20:29:04 2013  linear algebra at 0.0%, ETA 1388h20m
Thu Dec 12 20:30:29 2013  checkpointing every 30000 dimensions
Fri Dec 13 07:08:29 2013  lanczos error: submatrix is not invertible
Fri Dec 13 07:08:29 2013  lanczos halted after 3691 iterations (dim = 233377)
Fri Dec 13 07:08:29 2013  linear algebra failed; retrying...
Fri Dec 13 07:08:29 2013  commencing Lanczos iteration (4 threads)
Fri Dec 13 07:08:29 2013  memory use: 7589.9 MB
Fri Dec 13 07:13:02 2013  linear algebra at 0.0%, ETA 1458h 3m
Fri Dec 13 07:14:28 2013  checkpointing every 30000 dimensions

I've tried to filter rels with LBPA/LBPR 32/31 bit and matrix was built successfully. I'm not sure what matrix I have to use for LA (32/32 or 32/31).

Code:

32/31
Thu Dec 12 13:36:21 2013  commencing linear algebra
Thu Dec 12 13:36:27 2013  read 34428322 cycles
Thu Dec 12 13:37:23 2013  cycles contain 103186534 unique relations
Thu Dec 12 13:49:01 2013  read 103186534 relations
Thu Dec 12 13:52:06 2013  using 20 quadratic characters above 4294917296
Thu Dec 12 14:00:50 2013  building initial matrix
Thu Dec 12 14:23:30 2013  memory use: 14415.3 MB
Thu Dec 12 14:25:26 2013  read 34428322 cycles
Thu Dec 12 14:25:40 2013  matrix is 34428139 x 34428322 (10534.2 MB) with weight 3237080408 (94.02/col)
Thu Dec 12 14:25:40 2013  sparse part has weight 2348336605 (68.21/col)
Thu Dec 12 14:34:17 2013  filtering completed in 2 passes
Thu Dec 12 14:34:31 2013  matrix is 34409633 x 34409815 (10532.8 MB) with weight 3236388045 (94.05/col)
Thu Dec 12 14:34:31 2013  sparse part has weight 2348189355 (68.24/col)
Thu Dec 12 14:39:13 2013  matrix starts at (0, 0)
Thu Dec 12 14:39:26 2013  matrix is 34409633 x 34409815 (10532.8 MB) with weight 3236388045 (94.05/col)
Thu Dec 12 14:39:26 2013  sparse part has weight 2348189355 (68.24/col)
Thu Dec 12 14:39:26 2013  saving the first 48 matrix rows for later
Thu Dec 12 14:39:36 2013  matrix includes 64 packed rows
Thu Dec 12 14:39:43 2013  matrix is 34409585 x 34409815 (10166.1 MB) with weight 2598798670 (75.52/col)
Thu Dec 12 14:39:43 2013  sparse part has weight 2320882467 (67.45/col)
Thu Dec 12 14:39:43 2013  using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Thu Dec 12 14:44:49 2013  commencing Lanczos iteration

jasonp · 2013-12-13, 22:22

Is this on windows? Visual Studio builds are reported to work okay but IIRC building from source in MinGW64 still leads to these kinds of problems.

Looking more carefully at your log, the error takes a while to happen; are you sure the machine is stable? Is it overclocked at all?

sashamkrt · 2013-12-14, 15:31

It's on Windows 7, SVN 946 from http://www.mersenneforum.org/showpos...7&postcount=10

Yes, I'm sure. The machine is very stable it's working without any problem more than two years and it's not overclocked at all.

jasonp · 2013-12-14, 15:57

I don't think that sieving more will help; this is an issue with the linear algebra on 64-bit windows and I don't remember the current status. It's theoretically possible for the LA to fail on its own even if everything is working perfectly, but in practice the odds of that are very low and this would be the first occurrence I can remember. If it happens again, it's much more likely to be something to do with the binary you are using. Did you compile it yourself or use someone else's binary? If the former, what compiler was used?

sashamkrt · 2013-12-14, 18:02

The binary compiled by wombatman
---
Compiling a 64-bit MSieve in Visual Studio has gotten me past the error point and allowed linear algebra to start. The file is attached. It was compiled on a Corei7 processor without ECM or CUDA. It's basically just for doing the -nc step.
---

I'll look for the progress on Monday and try the different build if this error will remain.

sashamkrt · 2013-12-16, 04:10

Checked it. No new errors from Friday. 5.2% done. I hope there will be no more errors.

2013-12-16, 04:10	#6
sashamkrt Aug 2013 11001₂ Posts	Checked it. No new errors from Friday. 5.2% done. I hope there will be no more errors. Last fiddled with by sashamkrt on 2013-12-16 at 04:11

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Resume linear algebra	Timic	Msieve	35	2020-10-05 23:08
Has anyone tried linear algebra on a Threadripper yet?	fivemack	Hardware	3	2017-10-03 03:11
restarting nfs linear algebra	cubaq	YAFU	2	2017-04-02 11:35
Linear algebra at 600%	CRGreathouse	Msieve	8	2009-08-05 07:25
Linear algebra proof	Damian	Math	8	2007-02-12 22:25

2013-12-13, 22:22	#2
jasonp Tribal Bullet Oct 2004 3,541 Posts	Is this on windows? Visual Studio builds are reported to work okay but IIRC building from source in MinGW64 still leads to these kinds of problems. Looking more carefully at your log, the error takes a while to happen; are you sure the machine is stable? Is it overclocked at all?

2013-12-14, 15:31	#3
sashamkrt Aug 2013 5² Posts	It's on Windows 7, SVN 946 from http://www.mersenneforum.org/showpos...7&postcount=10 Yes, I'm sure. The machine is very stable it's working without any problem more than two years and it's not overclocked at all.

2013-12-14, 15:57	#4
jasonp Tribal Bullet Oct 2004 3,541 Posts	I don't think that sieving more will help; this is an issue with the linear algebra on 64-bit windows and I don't remember the current status. It's theoretically possible for the LA to fail on its own even if everything is working perfectly, but in practice the odds of that are very low and this would be the first occurrence I can remember. If it happens again, it's much more likely to be something to do with the binary you are using. Did you compile it yourself or use someone else's binary? If the former, what compiler was used?

2013-12-14, 18:02	#5
sashamkrt Aug 2013 31₈ Posts	The binary compiled by wombatman --- Compiling a 64-bit MSieve in Visual Studio has gotten me past the error point and allowed linear algebra to start. The file is attached. It was compiled on a Corei7 processor without ECM or CUDA. It's basically just for doing the -nc step. --- I'll look for the progress on Monday and try the different build if this error will remain.