msieve help

[Batalov]

He-he. That's exactly why I say 90M is not enough. About 92M (minimum) is needed in my experience. You have re-discovered the cusp of convergence.

You need a few more relations, 2-3Ms.

minimums appear to be:
46M for 29/29
92M for 30/30
18xM for 31/31

Right now, I have a slightly over the "cusp of convergence" matrix for a 31/30M with 135M relations... It barely converged and the matrix is a monster (14M; expected 11-12M). The matrix will be much better with maybe 10M more relns ...but we are not going to sieve them: We are masochists.

[em99010pepe]

Thank you all.

Now my issue is other, the client uses 5.9 GB of memory and therefore the machine stays very slow. Is there a flag to limit the memory used?

Carlos

EDIT:
-mb <num> # of megabytes of memory for postprocessing
(set automatically if unspecified or zero)

[10metreh]

Originally Posted by em99010pepe

Thank you all.

Now my issue is other, the client uses 5.9 GB of memory and therefore the machine stays very slow. Is there a flag to limit the memory used?

Carlos

If this is in linear algebra, you have to sieve some more, which will give you a matrix that uses less memory.

[em99010pepe]

Originally Posted by 10metreh

If this is in linear algebra, you have to sieve some more, which will give you a matrix that uses less memory.

So memory used is dependent of the matrix size?

[fivemack]

Yes; the whole of that 'matrix is 11726538 x 10890468 (3500.8 MB) with weight 1019639386 (93.63/col)' has to be read through twice for each of the 130,000 iterations of the block Lanczos algorithm, so it really has to be stored in memory; about six to tenvectors of 10.9 million 64-bit numbers need to be stored in memory too.

I generally don't expect to do anything else on a machine that's running the msieve matrix step.

The really weird behaviour in the first couple of runs reminds me of things I've seen when I didn't have enough disc space; msieve uses quite a lot of disc space, enough space for a second copy of msieve.dat is definitely enough but anything vastly less than that isn't safe, and does tend to behave strangely rather than reporting 'disc full'.

[em99010pepe]

Originally Posted by fivemack

Yes; the whole of that 'matrix is 11726538 x 10890468 (3500.8 MB) with weight 1019639386 (93.63/col)' has to be read through twice for each of the 130,000 iterations of the block Lanczos algorithm, so it really has to be stored in memory; about six to tenvectors of 10.9 million 64-bit numbers need to be stored in memory too.

I generally don't expect to do anything else on a machine that's running the msieve matrix step.

The really weird behaviour in the first couple of runs reminds me of things I've seen when I didn't have enough disc space; msieve uses quite a lot of disc space, enough space for a second copy of msieve.dat is definitely enough but anything vastly less than that isn't safe, and does tend to behave strangely rather than reporting 'disc full'.

I really use the machine, it is not a dedicated cruncher. Msieve is running on my second hard disk, a 80 GB disk, so I suppose that's enough.
Anyway, Greg will do more sieving to make the matrix smaller. Let's see what happens. Meanwhile I switched the cores to Paul's ecmnet server.

[Andi_HB]

Hey frmky that`s a huge available Ram - wow
estimated available RAM is 64454.2 MB

[frmky]

Originally Posted by Andi_HB

Hey frmky that`s a huge available Ram - wow
estimated available RAM is 64454.2 MB

That's our 8-cpu Opteron quad-core system. It has 2 GB/core, so 64 GB of memory.

[jasonp]

Originally Posted by em99010pepe

Now my issue is other, the client uses 5.9 GB of memory and therefore the machine stays very slow. Is there a flag to limit the memory used?

-mb <num> # of megabytes of memory for postprocessing
(set automatically if unspecified or zero)

That setting is meant to control the memory use of NFS filtering. The linear algebra will eat as much memory as it needs, and in fact the initial stages behave worse in the latest version because large primes > 2^32 can be handled.

[em99010pepe]

Originally Posted by jasonp

That setting is meant to control the memory use of NFS filtering. The linear algebra will eat as much memory as it needs, and in fact the initial stages behave worse in the latest version because large primes > 2^32 can be handled.

Ok, thank you for your explanation. Thank you all for helping me, I really appreciate.

[henryzz]

whats the minimum the filtering can be set to use?