Filtering Error
 

Anyone seen an error like this?

nfs: commencing msieve filtering
23224469611142170527533931799771364948921791873477651501897003326195130809543391
91343395482994279284683341641166356193019531251152423567

commencing relation filtering
estimated available RAM is 2048.0 MB
commencing duplicate removal, pass 1
read 10M relations
read 20M relations
found 2856559 hash collisions in 22404052 relations
added 125 free relations
commencing duplicate removal, pass 2
found 1658597 duplicates and 20745580 unique relations
memory use: 82.6 MB
reading ideals above 10289152
commencing singleton removal, initial pass
failed to reallocate 655360000 bytes

msieve then stopped.... Any thoughts appreciated!

It means you ran out of memory, a reallocation of 655 MB failed.

Thanks Dubslow. I will rerun filtering after a reboot.

[QUOTE=Dubslow;392470]It means you ran out of memory, a reallocation of 655 MB failed.[/QUOTE]

If msieve is only using 82MB to filter, consistent with a small job like this, why is it trying to allocate 600MB? Hrmph.

[QUOTE=VBCurtis;392485]If msieve is only using 82MB to filter, consistent with a small job like this, why is it trying to allocate 600MB? Hrmph.[/QUOTE]

Rather the better question. :smile:

The duplicate removal is quite memory efficient. The first singleton removal pass is one of the two most memory intensive parts of the whole algorithm (the other being the initial matrix build). It is a bit suspicious that a dataset with 20M relations requires so much memory in that pass, but not impossible.

32-bit machine? That would run out of VM for a single process quickly.

Jason is right. The virtual memory problem was on my work machine which is 32 bit. It ran fine on my home 64 bit laptop. It was the smallest matrix that I have had memory problems on my 32 bit machine.

We could change the title of this forum to "Jason is right." :tu:

Jeez, no pressure there...