Msieve v1.47 feedback

[Andi47]

Originally Posted by EdH

Actually, at this point, things have gotten even worse - the hard drive has crashed, such that the machine can't even find it. The good part is that nothing of great value was on it.

I'll see how another drive and maybe a different OS work... more later...

Thanks again.

*ouch*

maybe the hard disk has been faulty from the beginning and thus resulting in endless loops? (I don't think that a (possible) software bug can crash a HDD beyond repair...)

[EdH]

Originally Posted by Andi47

*ouch*

maybe the hard disk has been faulty from the beginning and thus resulting in endless loops? (I don't think that a (possible) software bug can crash a HDD beyond repair...)

I agree that the software "shouldn't" have cause the failure, but the impending failure probably didn't cause the loop, either. Unfortunately, I have two other machines that display the same non-stop poly selection. One is Ubuntu and the other is Fedora. This makes them totally unusable for Aliqueit. I may have to edit my local copy of msieve to use wall time instead of CPU time, since this seems to be an issue with linux and I'm compiling my own here anyway. These machines will be working on composites in the range of 100 digits +/- 10, or so, so using wall time set to about 30 minutes should work out fine.

[EdH]

Here's my "simple" solution to the never-ending poly run on my linux machines (in case there's interest):

1. edited aliqueit.ini to change

use_msieve_polyfind = true

to

use_msieve_polyfind = false

2. edited line 363 in factmsieve.py to add the -d switch jasonp suggested in the v1.46 feedback thread

args = ('-s {0:s} -l {1:s} -i {2:s} -nf {3:s} '

to

args = ('-d 30 -s {0:s} -l {1:s} -i {2:s} -nf {3:s} '

Thanks jasonp!

Note: The machine with bad HD has been brought back up with a different drive, onto which I installed a fresh Ubuntu 10.04 OS and all the files necessary to run aliqueit. I followed my page: Steps to install and set up Aliqueit on an Ubuntu computer to perform the installation. It is now working on sequence 60384 as a test.

[EdH]

Thanks for the editing, jasonp. I have another change, though.

My 30 minutes has proven too short for the LA step with a recent c102. I have subsequently increased the value to 45 and all seems well for my current Aliqueit sequences. However, I'm sure if I were to tackle anything greater than 110 digits, I would run the risk of again cutting the LA off prior to completion. But, for now, I'm using this:

args = ('-d 45 -s {0:s} -l {1:s} -i {2:s} -nf {3:s} '

(I suppose, to implement this properly, I'm going to have to do some coding in factmsieve.py such that the -d option is only inserted during the poly search. Maybe, later. . .)

Sorry to be a pain.

Thanks for all.

[Brian Gladman]

Hi EdH,

You should be able to add the extra parameter for MSIEVE at the point at which run_msieve is _called_ rather than within its definition. If you can say where run_msieve is being called from (i.e. where you want the extra parameter), I will add an option to set this value.

Brian

[EdH]

Originally Posted by Brian Gladman

Hi EdH,

You should be able to add the extra parameter for MSIEVE at the point at which run_msieve is _called_ rather than within its definition. If you can say where run_msieve is being called from (i.e. where you want the extra parameter), I will add an option to set this value.

Brian

Thanks Brian,

Basically, my linux machines don't resolve CPU time correctly for the msieve poly search, so it doesn't end, at least not in a reasonable time. Therefore, the poly search needs a hard "wall" time built in via the -d switch. I chose to put it in the general call args, but that interferes with the LA for larger composites. If you could place the additional switch such that it only adds the wall time (-d ##) option for the poly select, that would be great.

Ed

[Brian Gladman]

Hi Ed,

Could you please check the attached version when you get an opportunity?

You will need to set MSIEVE_POLY_TIME_LIMIT (near the start of the file) to the value you want before running it.

Brian

[EdH]

Originally Posted by Brian Gladman

Hi Ed,

Could you please check the attached version when you get an opportunity?

You will need to set MSIEVE_POLY_TIME_LIMIT (near the start of the file) to the value you want before running it.

Brian

Thanks Brian,

Preliminary tests are a success. I'll know more in a few days.

Ed

[EdH]

After a couple of days, all seems fine.

I have v75 running on five machines and all are working well. Two are WinXP and three are Ubuntu 10.04. I might also fire up a Fedora 13 for a short test. . .

Thanks, Brian!

Ed

[Jeff Gilchrist]

This is interesting, I'm doing a MPI LA and am seeing quite different speeds on Gigabit vs InfiniBand but not how I expected.

Using a 5x5 grid, after 0.9% completion I get:
Gigabit (Xeon 2.93GHz): 141 hours to complete
InfiniBand (Xeon 2.8GHz): 171 hours to complete

The InfiniBand job is spread on 4 nodes (8 cores each) while the Gigabit job is spread on 12 nodes (8 cores each). Do you think I'm getting slower performance on the InfiniBand job because I'm running into memory bus contention with more MPI processes running on the same nodes?

[frmky]

Originally Posted by Jeff Gilchrist

This is interesting, I'm doing a MPI LA and am seeing quite different speeds on Gigabit vs InfiniBand but not how I expected.

Using a 5x5 grid, after 0.9% completion I get:
Gigabit (Xeon 2.93GHz): 141 hours to complete
InfiniBand (Xeon 2.8GHz): 171 hours to complete

The InfiniBand job is spread on 4 nodes (8 cores each) while the Gigabit job is spread on 12 nodes (8 cores each). Do you think I'm getting slower performance on the InfiniBand job because I'm running into memory bus contention with more MPI processes running on the same nodes?

This is actually more complicated than it first appears. For Core 2 class Xeons using DDR2, I find lower runtimes using only four of the eight cores. The optimum seems to be using two MPI processes per node and two threads per process. I haven't had the opportunity to test it on Core i7 class Xeons or the DDR3-based Opterons.