Filtering Error

richs · 2015-01-15, 01:03

Anyone seen an error like this?

nfs: commencing msieve filtering
23224469611142170527533931799771364948921791873477651501897003326195130809543391
91343395482994279284683341641166356193019531251152423567

commencing relation filtering
estimated available RAM is 2048.0 MB
commencing duplicate removal, pass 1
read 10M relations
read 20M relations
found 2856559 hash collisions in 22404052 relations
added 125 free relations
commencing duplicate removal, pass 2
found 1658597 duplicates and 20745580 unique relations
memory use: 82.6 MB
reading ideals above 10289152
commencing singleton removal, initial pass
failed to reallocate 655360000 bytes

msieve then stopped.... Any thoughts appreciated!

Dubslow · 2015-01-15, 01:11

It means you ran out of memory, a reallocation of 655 MB failed.

richs · 2015-01-15, 05:30

Thanks Dubslow. I will rerun filtering after a reboot.

VBCurtis · 2015-01-15, 06:12

Quote:

Originally Posted by Dubslow

It means you ran out of memory, a reallocation of 655 MB failed.

If msieve is only using 82MB to filter, consistent with a small job like this, why is it trying to allocate 600MB? Hrmph.

Dubslow · 2015-01-16, 01:06

Quote:

Originally Posted by VBCurtis

If msieve is only using 82MB to filter, consistent with a small job like this, why is it trying to allocate 600MB? Hrmph.

Rather the better question.

jasonp · 2015-01-16, 01:28

The duplicate removal is quite memory efficient. The first singleton removal pass is one of the two most memory intensive parts of the whole algorithm (the other being the initial matrix build). It is a bit suspicious that a dataset with 20M relations requires so much memory in that pass, but not impossible.

32-bit machine? That would run out of VM for a single process quickly.

richs · 2015-01-18, 01:47

Jason is right. The virtual memory problem was on my work machine which is 32 bit. It ran fine on my home 64 bit laptop. It was the smallest matrix that I have had memory problems on my 32 bit machine.

VBCurtis · 2015-01-18, 17:39

We could change the title of this forum to "Jason is right."

jasonp · 2015-01-18, 17:40

Jeez, no pressure there...

2015-01-15, 01:03	#1
richs "Rich" Aug 2002 Benicia, California 519₁₆ Posts	Filtering Error Anyone seen an error like this? nfs: commencing msieve filtering 23224469611142170527533931799771364948921791873477651501897003326195130809543391 91343395482994279284683341641166356193019531251152423567 commencing relation filtering estimated available RAM is 2048.0 MB commencing duplicate removal, pass 1 read 10M relations read 20M relations found 2856559 hash collisions in 22404052 relations added 125 free relations commencing duplicate removal, pass 2 found 1658597 duplicates and 20745580 unique relations memory use: 82.6 MB reading ideals above 10289152 commencing singleton removal, initial pass failed to reallocate 655360000 bytes msieve then stopped.... Any thoughts appreciated!

2015-01-18, 01:47	#7
richs "Rich" Aug 2002 Benicia, California 3²·5·29 Posts	Jason is right. The virtual memory problem was on my work machine which is 32 bit. It ran fine on my home 64 bit laptop. It was the smallest matrix that I have had memory problems on my 32 bit machine. Last fiddled with by richs on 2015-01-18 at 01:49 Reason: Added final comment

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
NFS filtering error...	Stargate38	YAFU	4	2016-04-20 16:53
The big filtering bug strikes again (I think)	Dubslow	Msieve	20	2016-02-05 14:00
Filtering	Sleepy	Msieve	25	2011-08-04 15:05
Filtering	R.D. Silverman	Cunningham Tables	14	2010-08-05 08:30
Filtering Phenomenon	R.D. Silverman	NFSNET Discussion	2	2005-09-16 04:00

2015-01-15, 01:11	#2
Dubslow Basketry That Evening! "Bunslow the Bold" Jun 2011 40<A<43 -89<O<-88 3·29·83 Posts	It means you ran out of memory, a reallocation of 655 MB failed.

2015-01-15, 05:30	#3
richs "Rich" Aug 2002 Benicia, California 3²·5·29 Posts	Thanks Dubslow. I will rerun filtering after a reboot.

2015-01-16, 01:28	#6
jasonp Tribal Bullet Oct 2004 3,541 Posts	The duplicate removal is quite memory efficient. The first singleton removal pass is one of the two most memory intensive parts of the whole algorithm (the other being the initial matrix build). It is a bit suspicious that a dataset with 20M relations requires so much memory in that pass, but not impossible. 32-bit machine? That would run out of VM for a single process quickly.

2015-01-18, 17:39	#8
VBCurtis "Curtis" Feb 2005 Riverside, CA 11375₈ Posts	We could change the title of this forum to "Jason is right."

2015-01-18, 17:40	#9
jasonp Tribal Bullet Oct 2004 3,541 Posts	Jeez, no pressure there...