I am having trouble getting msieve threads to pass on into the openmpi processes, especially -t 4. All of the following are using the same instance of mpi-aware msieve. All these observations are after waiting for the LA to get past the ETA messages by a bit.

If I run msieve with -t 4, without calling it via mpirun, using top, I can see one instance of msieve using ~350% of my quad core CPU:

Code:

../msieve/msieve -i number.ini -s number.dat -l number.log -nf number.fb -t 4 -nc2

If I then run the same command via mpi, top shows one instance of msieve running at <=100% and the ETA is appropriately longer.

Code:

mpirun -np 1 --hostfile ./mpi_hosts111 ../msieve/msieve -i number.ini -s number.dat -l number.log -nf number.fb -t 4 -nc2 1,1

If, still using one machine, I change to 2 mpi processes and -t 2, top will show 2 instances, both are <= 100%.

Code:

mpirun -np 2 --hostfile ./mpi_hosts221 ../msieve/msieve -i number.ini -s number.dat -l number.log -nf number.fb -t 2 -nc2 2,1

If I add in a second machine, top shows 2 instances on both machines with all instances showing <=150%.

Code:

mpirun -np 4 --hostfile ./mpi_hosts221 ../msieve/msieve -i number.ini -s number.dat -l number.log -nf number.fb -t 2 -nc2 4,1

If I then adjust to 2 mpi processes with -t 4, top changes to one instance on each machine, but <=100% for both.

Code:

mpirun -np 2 --hostfile ./mpi_hosts111 ../msieve/msieve -i number.ini -s number.dat -l number.log -nf number.fb -t 4 -nc2 2,1

Note, that the mpi_hosts' suffix is representative of the slots to use per three machines.

It appears that -t 2 works with multiple machines, but -t 4 will not work at all via mpi.

Any thoughts on the above observations are welcome...