Msieve v1.47 feedback

[jasonp]

The Teragrid cluster we've been using, with quad-core nodes, is going to be replaced at the end of the year with much more powerful nodes (12 cores each instead of 4 now). This is actually bad for us, because we get charged per-core for the CPU time we use, not per-node. So the new cluster will cost 3x as much in CPU time but will not be 3x faster for our application.

[Jeff Gilchrist]

Originally Posted by frmky

This is actually more complicated than it first appears. For Core 2 class Xeons using DDR2, I find lower runtimes using only four of the eight cores. The optimum seems to be using two MPI processes per node and two threads per process. I haven't had the opportunity to test it on Core i7 class Xeons or the DDR3-based Opterons.

Yes, so many things that can change the performance. I had to re-start the process about half way through and now they have switched, 69h left on the Infiniband and 81 h left on the gigabit from the same checkpoint. It all depends on the mix and on a busy cluster I don't have the choice of hand picking the ultimate layout.

Jeff.

[EdH]

I've been running Aliqueit on one of my linux boxes and a recent (now getting old) qs procedure seems to be taking a bit long for a c96. At least, I kind of think it is taking a bit long - it's been running it for over 72 hours:

[Dec 03 2010, 20:53:00] c96: running qs (msieve)...
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sieving in progress (press Ctrl-C to pause)
81521 relations (20398 full + 61123 combined from 1207085 partial), need 81272
sieving complete, commencing postprocessing

This particular machine has performed qs on two c94s in 08:22:54 and 11:26:00 and a c97 in 16:11:29. Top seems to show msieve as still working along, at around 90% CPU and 0.5% memory. It is running at normal priority.

Is there any info I should gather prior to killing the current run?

In checking this out, I found that this was the first calculation of a new run of Aliqueit on a c105 in sequence 120760, if that is of any significance. Msieve was compiled from the repository, but seems to only be showing version 1.47. I will update and recompile after I resolve the current issue.

[jasonp]

If it's stuck in the postprocessing, I'd suspect bad hardware or a buggy compiler. How big is the relation file? If it's under 1GB maybe I can download the relations and take a look.

[EdH]

Originally Posted by jasonp

If it's stuck in the postprocessing, I'd suspect bad hardware or a buggy compiler. How big is the relation file? If it's under 1GB maybe I can download the relations and take a look.

If you're interested, the file is available at msieve,dat. I'll leave it there for a couple days. It's 42.8MB. It just passed 86 hours and I'll probably just stop and restart to see what happens, unless you'd like anything else prior to interrupting it.

It is possible I don't have enough memory in this machine. I just checked and it has only 256MB. I now remember only having that much of the right type, when I set it up.

[em99010pepe]

I got this error while running Msieve v. 1.48:

error: corrupt state, please restart from checkpoint

I restarted the LA. Let's see what happens in the next hours.

Carlos