Msieve 1.50 feedback

[jrk]

Originally Posted by poily

I'll play with the GPU mode later after jrk sends me the patches he promised.

Check your messages.

[Batalov]

Originally Posted by Batalov

The question when to bail everyone decides for themselves - there are too many parameters (how many sievers will be used, how many poly select threads are used, etc). Or leave it up to the built-in timeout...

It occurred to me that one change that can be very easily added (and is probably about right) is
to multiply those expected values by another 1.15 -- #ifdef HAVE_CUDA.

[jrk]

Originally Posted by jrk

Check your messages.

For the record, the problem that poily & I identified was a discrepancy in pointer sizes between the msieve binary and the PTX assembly on his macbook. The msieve binary was built for 64bit and the PTX for 32bit and this resulted in an overflow in the CUDA parameter assignment, which led to the reported error.

The fix was to add -m 64 to the nvcc build options, which is apparently necessary on the mac platform. I'm not sure what is the most proper way to handle this in the Makefile.

[jasonp]

Looks like this project is starting to really hurt from not having an autoconf script. I've been desperately avoiding trying to learn how the autotools work, but the contortions to avoid them are starting to exceed the pain of learning.

[Batalov]

You could go really fancy and redress it as a CMake project.
I know that that's what Illumina did to their sequencing pipeline. Looks trendy.

[debrouxl]

On the one hand, definitions for CMake are usually somewhat easier to create than definitions for autotools.
On the other hand, CMake is much less widely used, and doesn't support a range of platforms, functions, quirks and workarounds as wide as the autotools do.

[xilman]

I've a matrix which is 3564431 x 3564656 x 251590133 (70.58/col). Installed is stock OpenMPI on a Fedora 15 system.

Running "msieve -nc2 -t 8 ..." on a dual 4-core machine gives a predicted time of 15h38m --- in line with previous experience with other matrices of comparable size.

Using "mpirun -np 8 msieve -nc2 2x4 ..." on the same machine predicts 17h34m --- quite significantly longer. I would hope that performance would be better or at least comparable but perhaps I've misunderstood others' postings here.


What, if anything, am I likely to be doing wrong?


(Incidentally, I found out the hard way that setting -t 8 and using mpirun leads to a messy crash. I wasn't trying to use multithreading in this run, just failed to cut and paste correctly.)


Paul

[jasonp]

Did you mean '2,4' and not '2x4'? The CADO tools take the latter format but msieve should not. Anyway, 8 processes are kind of a lot for a matrix of that size, does 2x2 do any better?

Finally, Tom reported in his blog that there are some nasty contortions needed to assign process affinity when the processes are generated through a script like mpirun.

[fivemack]

Try something like

nohup mpirun -n 8 --bind-to-core --report-bindings numactl -l msieve -v -nc2 2,4 >> aus 2>> err

(the numactl -l might be unnecessary, it's to ensure that the jobs allocate memory only on the NUMA partition that they're bound to; they should be doing that by default). And ensure that the machine is otherwise idle, because (as you know) starting new jobs on any processor that's running an MPI-slice slows the whole job down - I found that I even needed to move hourly cronjobs off my big machine.

My timings for a 1M matrix on a machine that you might recognise are in

http://www.mersenneforum.org/showpos...5&postcount=65

[xilman]

Originally Posted by fivemack

Try something like

nohup mpirun -n 8 --bind-to-core --report-bindings numactl -l msieve -v -nc2 2,4 >> aus 2>> err

(the numactl -l might be unnecessary, it's to ensure that the jobs allocate memory only on the NUMA partition that they're bound to; they should be doing that by default). And ensure that the machine is otherwise idle, because (as you know) starting new jobs on any processor that's running an MPI-slice slows the whole job down - I found that I even needed to move hourly cronjobs off my big machine.

My timings for a 1M matrix on a machine that you might recognise are in

http://www.mersenneforum.org/showpos...5&postcount=65

Thanks. (And thanks also to jasonp for pointing out my typo and suggesting that 4 processors may be more appropriate than 8).

I'll let the current run complete and then conduct experiments based on Tom's advice.

Paul

[xilman]

Originally Posted by xilman

I'll let the current run complete and then conduct experiments based on Tom's advice.

Finished successfully and factors found on first dependency.

Tue Apr 17 10:47:26 2012  linear algebra at 0.0%, ETA 17h 5m
Tue Apr 17 10:47:35 2012  checkpointing every 210000 dimensions
Wed Apr 18 04:34:02 2012  lanczos halted after 56365 iterations (dim = 3564427)
Wed Apr 18 04:34:08 2012  recovered 35 nontrivial dependencies
Wed Apr 18 04:34:08 2012  BLanczosTime: 64456
Wed Apr 18 04:34:08 2012  elapsed time 17:54:17

Now to start experimenting.