lanczos error: submatrix is not invertible

Xyzzy · 2020-10-17, 21:21

Code:

Sat Oct 17 12:07:31 2020  lanczos error: submatrix is not invertible
Sat Oct 17 12:07:31 2020  lanczos halted after 180668 iterations (dim = 11424815)
Sat Oct 17 12:07:31 2020  linear algebra failed, aborting

We had this error today after we paused (CTRL+Z) the job and then resumed it. The error caused the job to stop.

We were able to restart the job so hopefully the work is intact?

jasonp · 2020-10-20, 18:21

How long between stop and resume? Also is VBITS set to > 64?

The error check in the linear algebra will always fail if you create checkpoints within three iterations of each other. I wonder if fast-forwarding the wall-clock time caused a checkpoint too early...

Xyzzy · 2020-10-21, 13:03

Quote:

Originally Posted by jasonp

How long between stop and resume?

Maybe an hour?

Quote:

Originally Posted by jasonp

Also is VBITS set to > 64?

It is whatever the default is.

Do you think the job is corrupted? We are at 44% with ~350 hours to go.

jasonp · 2020-10-21, 17:13

If you restarted from checkpoint and it got past the failure point then you should hopefully be able to finish. Maybe the failure wasn't related to getting suspended. Cosmic ray? Memory corruption?

RichD · 2020-10-21, 18:46

I've had that error once or twice. Restarting from a checkpoint completed with no further problems. I wrote it off as old hardware or thermal issue.

2020-10-17, 21:21	#1
Xyzzy "Mike" Aug 2002 5·1,597 Posts	lanczos error: submatrix is not invertible Code: Sat Oct 17 12:07:31 2020 lanczos error: submatrix is not invertible Sat Oct 17 12:07:31 2020 lanczos halted after 180668 iterations (dim = 11424815) Sat Oct 17 12:07:31 2020 linear algebra failed, aborting We had this error today after we paused (CTRL+Z) the job and then resumed it. The error caused the job to stop. We were able to restart the job so hopefully the work is intact?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
lanczos error: submatrix is not invertible	pinhodecarlos	Msieve	3	2016-12-10 10:40
"lanczos error: only trivial dependencies found" with massive oversieving	eigma	Msieve	21	2015-05-28 03:27
lanczos error: submatrix is not invertible	Mini-Geek	Msieve	6	2014-02-13 14:38
Msieve with MPI block Lanczos	jasonp	Msieve	104	2013-06-17 11:28
Lanczos error	Andi47	Msieve	7	2009-01-11 19:33

2020-10-20, 18:21	#2
jasonp Tribal Bullet Oct 2004 DCE₁₆ Posts	How long between stop and resume? Also is VBITS set to > 64? The error check in the linear algebra will always fail if you create checkpoints within three iterations of each other. I wonder if fast-forwarding the wall-clock time caused a checkpoint too early...

2020-10-21, 17:13	#4
jasonp Tribal Bullet Oct 2004 2×3×19×31 Posts	If you restarted from checkpoint and it got past the failure point then you should hopefully be able to finish. Maybe the failure wasn't related to getting suspended. Cosmic ray? Memory corruption?

2020-10-21, 18:46	#5
RichD Sep 2008 Kansas 19×173 Posts	I've had that error once or twice. Restarting from a checkpoint completed with no further problems. I wrote it off as old hardware or thermal issue.