mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Software

Reply
 
Thread Tools
Old 2012-06-26, 03:42   #1
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

887610 Posts
Default P95: Sequential (?!) Stage 2 of P-1?

Short version:

How can I convince P95 to run only one stage 2 of P-1 at a time? If there are more (i.e. two or more workers want to do P-1 stage 2 in the same time), he should ignore all except one, and of course, the "ignored" workers would not sleep, but have to move forward doing P-1 stage 1 for the next exponents in their queues, and later come back to finish P-1 of the ignored expos. What is the best way to do this?

Long version:

I am now "tuning" a wannabe industrial PC, with i7 620M and 4gigs of RAM, from the "thermal" point of view (no fans, passive cooling, heat pipes and aluminum blocks). Because I don't want to waste the CPU resources, I gave him a couple of P-1 to do, on all 4 cores (2 physical, with HT). P-1 assignments have the advantage that are finished fast, and if someone considers that all possible tests are done, and decides to format the SSD in the middle of the work, then it is not a big loss, at most a day or two of wasted work.

The toy went through Stage_1 of all 4 expos quite rapidly (faster then expected), and started Stage_2 with ALL 4 expos in the same time. Here the boils started to burst. I was allocating 2048 megs for P95, from the 4096 total mem.

I have no idea (yet) how much Win7-basic-32bits (the "free" version) need for itself. Sysinfo says "4.00GB installed. 3.05GB usable". The limitation come from the 32bits of the OS, and most probable the "usable" includes the OS. That is, in fact, the system can only see and use 3.05G of memory, so there may be less then 2G for the user.

I started to wonder why there is no result reported after about 24 hours, as he was supposed to finish Stage_2 in less then this time. When I came to the office I checked the logs. He was spending the last 20 hours with "Memory allocation error - try again with lower memory" but sometime it was trying with lower memory, sometime with higher. At the end it went in a very nice interlocked loop where all 4 tasks were allocating/dellocating memory and nothing was progressing, like in the classical problems from my university (years ago), with semaphores and philosophers eating rice with chopsticks. When one worker was deallocating memory, other workers jump to grab it. The laziest one of course could not find it anymore, and started screaming the error message. And the process repeat. The temperature went down to 60 during all this process (it was around 80-85 during stage 1, well... we still have to work those coolers... but this is different story).

That interlock was totally nerd. I stopped everything, with "stop" then "exit", as gracefully as I could, but in spite of this, I found out there were no saving files. He deleted the saving files, or corrupted them in some way that he could not reuse them. I re-arranged the "worktodo" file to have only two workers and modified the options of the program to use only 2 workers, with HT (one worker use both logical cores from a full physical one). I did not change the amount of allocated memory, it was still 2048 megs, which was probably too much, but that time I did not realize the mistake. Of course, as I said, he lost the Stage_1 files somehow in this "allocation error" process, and started everything from scratch, this time with two cores only. There was a day and there was a night, as in the bible, then this morning when I come to the office, my toy was trying to do - guess what? - allocating memory for stage 2 on two workers, very nice interlocked each-other.

This time I checked the temporary files and made backups before exiting. I cancelled the HT (let still two workers, but single core each) and reduced the day/night memory to 1024.

Started again. He found the temp file all right, and the good part was that he said "ignoring recommended B1 limit" (which was much lower for 1024 megs memory limit) and "using the limit in the saved file", which is much higher. And he proceed to stage two, of course with BOTH workers in the same time, and they argued each-other for about 10 minutes about the allocation of the memory.

Totally pain in the ass!! But I saw that the amount was decreasing every time, and I resisted the temptation to put a stop to it. It started with both workers trying to allocate 1022 megs, which was way too much, then decreasing 2-3 megs after every "memory allocation error, trying again with lower memory" error message. At least, it stabilizes at 500 megs for worker 1 and 522 megs for worker 2, and - god bless - started stage 2 at least.

Worker 1 using 18 primes per bunch (in this moment the third bunch 36 primes from 480 already done)
Worker 2 using 19 primes per bucket (in this moment the second bucket, 19 primes from 480 already done)

Now you see my problem, if you were quite patient to read all this crap.

I want to be able to explain to P95 that only ONE instance of P-1 stage 2 is allowed at any moment of time. And that instance should use about one gig of memory. The other workers are free to do anything else in this time, except waiting or fighting for memory. What is the best way to do this? The options in the undoc file are not exactly what I am looking for. Should I try to arrange the worktodo in such a way to avoid "multiple stage 2" in the same time? Or there is a much better solution which I can not see now?
LaurV is offline   Reply With Quote
Old 2012-06-26, 03:57   #2
NBtarheel_33
 
NBtarheel_33's Avatar
 
"Nathan"
Jul 2008
Maryland, USA

3·7·53 Posts
Default MaxHighMemWorkers in prime.txt

In prime.txt, set MaxHighMemWorkers = n, where n is the maximum number of memory intensive threads you would want running at any one time. This will stagger the Stage 1s and Stage 2s just as you were asking.
NBtarheel_33 is offline   Reply With Quote
Old 2012-06-26, 04:02   #3
Xyzzy
 
Xyzzy's Avatar
 
"Mike"
Aug 2002

22×3×643 Posts
Default

MaxHighMemWorkers=1
Xyzzy is offline   Reply With Quote
Old 2012-06-26, 04:33   #4
PageFault
 
PageFault's Avatar
 
Aug 2002
Dawn of the Dead

5·47 Posts
Default

To avoid the memory allocation error loops when using 32 bit Vindovs, don't specify 2048 MB - I had the same issue and dropping to 1920 MB allowed for the overhead and the ram used by LL on the second core.

BTW, you should see all 4 GB available - I'm on a much older 32 bit, Serber 2003, and I get all of the 4 GB installed.
PageFault is offline   Reply With Quote
Old 2012-06-26, 06:45   #5
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

100010101011002 Posts
Default

Thanks to everybody for the help. I might have been blind when I was reading the undoc.txt file. Now searching it for MaxHighMem bla bla, it is written BIG inside... I added it to the prime.txt and it seems to work (even the undoc said it should be in local.txt) as I wanted (or at least for now, it kicked out the second worker, who started to do stage 1 of the next expo, and the first worker took a bit more of memory). Good.

Regarding the "seeing all memory under 32 bit OS", this is still unclear how it should be possible, as the max address allowed by the OS is 0xC0000000. This document still have some errors in it. All WinXP_32 will not show more then 3Gigs, even if you have 4, 8, etc. and you CANNOT use the rest. This problem seems not to be solved in Win7_32 bits, at least not in the basic versions (not Premium, Pro, Ultimate, but INCLUDE Home-Basic). I will ask some of my ppl here to install some Win7_64 on that toy later on the week, right now everybody is busy with other things.
LaurV is offline   Reply With Quote
Old 2012-06-26, 08:51   #6
NBtarheel_33
 
NBtarheel_33's Avatar
 
"Nathan"
Jul 2008
Maryland, USA

111310 Posts
Default

Quote:
Originally Posted by LaurV View Post
Thanks to everybody for the help. I might have been blind when I was reading the undoc.txt file. Now searching it for MaxHighMem bla bla, it is written BIG inside... I added it to the prime.txt and it seems to work (even the undoc said it should be in local.txt) as I wanted (or at least for now, it kicked out the second worker, who started to do stage 1 of the next expo, and the first worker took a bit more of memory). Good.

Regarding the "seeing all memory under 32 bit OS", this is still unclear how it should be possible, as the max address allowed by the OS is 0xC0000000. This document still have some errors in it. All WinXP_32 will not show more then 3Gigs, even if you have 4, 8, etc. and you CANNOT use the rest. This problem seems not to be solved in Win7_32 bits, at least not in the basic versions (not Premium, Pro, Ultimate, but INCLUDE Home-Basic). I will ask some of my ppl here to install some Win7_64 on that toy later on the week, right now everybody is busy with other things.
Search the forum for this problem. I believe there were some other folks who had 4 GB of RAM but Win32 couldn't use it (or see it) all. There really isn't much of a reason these days not to just go 64-bit, save for a few antiquated scanners and printers.
NBtarheel_33 is offline   Reply With Quote
Old 2012-06-26, 09:06   #7
axn
 
axn's Avatar
 
Jun 2003

4,723 Posts
Default

Quote:
Originally Posted by LaurV View Post
Regarding the "seeing all memory under 32 bit OS", this is still unclear how it should be possible, as the max address allowed by the OS is 0xC0000000. This document still have some errors in it. All WinXP_32 will not show more then 3Gigs, even if you have 4, 8, etc. and you CANNOT use the rest. This problem seems not to be solved in Win7_32 bits, at least not in the basic versions (not Premium, Pro, Ultimate, but INCLUDE Home-Basic). I will ask some of my ppl here to install some Win7_64 on that toy later on the week, right now everybody is busy with other things.
The OS should be able to see and use all 4GB. An individual process can see only 2GB (or 3GB if compiled with large_address_aware + boot.ini config). See: http://msdn.microsoft.com/en-us/libr...=vs.85%29.aspx
axn is online now   Reply With Quote
Old 2012-06-27, 01:53   #8
PageFault
 
PageFault's Avatar
 
Aug 2002
Dawn of the Dead

5·47 Posts
Default

You have to enable Physical Address Extension and it will show all, even large amounts. With sever 2003 this was done by default at install time.

Quote:
Originally Posted by LaurV View Post
Regarding the "seeing all memory under 32 bit OS", this is still unclear how it should be possible, as the max address allowed by the OS is 0xC0000000. This document still have some errors in it. All WinXP_32 will not show more then 3Gigs, even if you have 4, 8, etc. and you CANNOT use the rest. This problem seems not to be solved in Win7_32 bits, at least not in the basic versions (not Premium, Pro, Ultimate, but INCLUDE Home-Basic). I will ask some of my ppl here to install some Win7_64 on that toy later on the week, right now everybody is busy with other things.
PageFault is offline   Reply With Quote
Old 2012-06-27, 08:04   #9
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

887610 Posts
Default

Some of you confuse /PAE, /4GT, /DEP, /AWE, /3GB, virtual memory, physical memory, etc. Read more about these things.

Fact: you can not get windows 32 bits to use all of 4GB of memory on CPU's with 32 lines of physical addresses. /PAE enables some CPUs to switch 4 of their output pins from "whatever else general purpose i/o pin" to "address line", therefore having 36 physical address lines. On that systems you can get win32 to use 4GB or more, if you have it. This has nothing to do with being server or not server OS.
LaurV is offline   Reply With Quote
Old 2012-06-28, 04:10   #10
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

22·7·317 Posts
Default

Quote:
Originally Posted by LaurV View Post
MaxHighMem bla bla, I added it to the prime.txt and it seems to work (even the undoc said it should be in local.txt)
Grrr.. correction: it does not work, and I just lost another 5 hours in memory negotiations.... It has to be added in the local.txt, exactly as the undoc.txt explains... :surprised, and P95 has to be exited (from tools/exit). Stopping it (tools/stop) then restart (tools/continue) is not enough. Just to be clearly stated...

Last fiddled with by LaurV on 2012-06-28 at 04:12
LaurV is offline   Reply With Quote
Old 2012-06-28, 04:22   #11
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3·2,399 Posts
Default

Quote:
Originally Posted by LaurV View Post
Grrr.. correction: it does not work, and I just lost another 5 hours in memory negotiations.... It has to be added in the local.txt, exactly as the undoc.txt explains... :surprised, and P95 has to be exited (from tools/exit). Stopping it (tools/stop) then restart (tools/continue) is not enough. Just to be clearly stated...
At least as far as the latter goes, yes of course, we've all been giving that answer for a long time



(Pausing tests will not cause Prime95 to re-read the ini files.)
Dubslow is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
sequential number test Bundu Programming 20 2012-02-19 18:09
What's the largest known sequential prime? Unregistered Homework Help 4 2009-09-11 11:46
Stage 1 with mprime/prime95, stage 2 with GMP-ECM D. B. Staple Factoring 2 2007-12-14 00:21
Need help to run stage 1 and stage 2 separately jasong GMP-ECM 9 2007-10-25 22:32
Stage 1 and stage 2 tests missing Matthias C. Noc PrimeNet 5 2004-08-25 15:42

All times are UTC. The time now is 07:02.

Sat Oct 31 07:02:35 UTC 2020 up 51 days, 4:13, 2 users, load averages: 2.36, 1.79, 1.71

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.