Realloc in CWI Filter

[dsouza123]

Found further info on the full 4 GB for a 32-bit exe.

http://h20331.www2.hp.com/Hpsub/cach...0-225-121.html

Can a 32-bit application access more memory when running on XP x64?

In general no, most standard Windows 32-bit applications or processes are limited to 2GB of virtual address space, the same as in 32-bit XP. Windows 32-bit applications that have been built with the /LARGEADDRESSAWARE switch can access 4GB of RAM. The same application running on Windows XP 32-bit with Service Pack 2 is limited to 3GB of RAM so you gain 1GB of address space on XP x64.

Alpertron's does your post mean if the IMAGE_FILE_LARGE_ADDRESS_AWARE bit
is changed in an existing exe then the program can use the larger amount of memory ?

[alpertron]

Originally Posted by dsouza123

Alpertron's does your post mean if the IMAGE_FILE_LARGE_ADDRESS_AWARE bit
is changed in an existing exe then the program can use the larger amount of memory ?

You are right, but notice that setting that bit in the executable image is not enough. The program must not use signed compares between pointers because in this case it would not work in a > 2GB environment. It appears that some programs are written in this way (like casting pointers to integers and comparing them). These programs would hang if the IMAGE_FILE_LARGE_ADDRESS_AWARE is set and allocate more than 2 GB of memory.

[xilman]

Originally Posted by Prime95

How about adding a command line switch that lets you pick the initial amount of memory to malloc?

If realloc cannot grow the allocation in-place it would have to allocate the larger space and copy the data. During that time, you would have both the smaller and larger spaces allocated. I don't know if you are actually running into this problem.

That is precisely the problem Bob is experiencing.

[ewmayer]

I ran into similar problems (under Windows and the then-current version of Mac OS) several years back when doing multiple large-FFT-length self-tests runs of my Mlucas code. As Paul and George suggest, the best workaround (I hesitate to call it "solution") is to predetermine/specify the maximum amount of memory (or array dimension, or whatever) will be needed for the current task or series thereof and allocate that. It would be nice if more OSes had a dynamic-reallocate capability which would allow them on-the-fly to remap memory if a larger contiguous chunk is needed than is available with a current memory mapping - but simple as the concept sounds, we're not there yet.

[R.D. Silverman]

Originally Posted by ewmayer

I ran into similar problems (under Windows and the then-current version of Mac OS) several years back when doing multiple large-FFT-length self-tests runs of my Mlucas code. As Paul and George suggest, the best workaround (I hesitate to call it "solution") is to predetermine/specify the maximum amount of memory (or array dimension, or whatever) will be needed for the current task or series thereof and allocate that. It would be nice if more OSes had a dynamic-reallocate capability which would allow them on-the-fly to remap memory if a larger contiguous chunk is needed than is available with a current memory mapping - but simple as the concept sounds, we're not there yet.

The difficulty in pre-allocating for filter code is that it is hard to know
ahead of time how much will be needed....

[xilman]

Originally Posted by R.D. Silverman

The difficulty in pre-allocating for filter code is that it is hard to know ahead of time how much will be needed....

Which is why I suggested, in effect, to allocate as much space as your machine will supprt.

Any luck with booting with the /3gb switch?


Paul

[R.D. Silverman]

Originally Posted by xilman

Which is why I suggested, in effect, to allocate as much space as your machine will supprt.

Any luck with booting with the /3gb switch?


Paul

I work in a severely restricted environment. I do not have priviledges
to make changes to system files. Any such changes must be done by
a sysadmin with special priviledges. One must request the change,
justify the reason for the change, identify the project, etc. etc. -->
mucho paperwork

I just work around the problem. When the filter job fails, I just
change the input parameters so that the matrix gets compressed
"more slowly".

I am about 1/2 way done with filtering for 2,1454L.

2,1462L is sieving.