Need mprime configuration advice for P-1

[Mark Rose]

I've been away for a while, but with winter here, it's time to burn some joules. In the past I mainly focused CPUs on DC, but I understand mprime v30.8 can efficiently use higher memory systems for P-1.

I have five 4 core Skylake systems with 32 GB of memory. I also plan to recombobulate four 4 core Haswell systems also with 32 GB of memory. I can allocate 30 GB or so on each system. They're all a little (but not severely) memory bandwidth constrained when doing LL/PRP. They all support AVX2/FMA.

I don't have time to catch up on all the P-1 minutiae. What's the best way to configure these with regards to workers?

[kriesel]

Originally Posted by Mark Rose

I don't have time to catch up on all the P-1 minutiae. What's the best way to configure these with regards to workers?

For what you described, I think, 2 workers, Memory=30720 in local.txt for both day and night values. (Since that exceeds 90% you'll need to edit the local.txt file, not enter it from the prime95 GUI.) See https://mersenneforum.org/showthread.php?t=28038 for a coordinated effort to do P-1 on DC candidates which had poor P-1 bounds previously.

[preda]

Originally Posted by Mark Rose

I've been away for a while, but with winter here, it's time to burn some joules. In the past I mainly focused CPUs on DC, but I understand mprime v30.8 can efficiently use higher memory systems for P-1.

I have five 4 core Skylake systems with 32 GB of memory. I also plan to recombobulate four 4 core Haswell systems also with 32 GB of memory. I can allocate 30 GB or so on each system. They're all a little (but not severely) memory bandwidth constrained when doing LL/PRP. They all support AVX2/FMA.

I don't have time to catch up on all the P-1 minutiae. What's the best way to configure these with regards to workers?

32GB is a bit low RAM for efficient wavefront P-1 second-stage. Maybe you could merge RAM to create systems with 128GB (or more).

[kriesel]

Response is only logarithmic with ram amount. 24GiB is enough per George in prime95 / mprime.

Not all Skylake support 128GiB ram.

[nordi]

One key parameter is the exponents your are working on. Setup for 100k exponents is different than for 10M exponents, but you didn't mention what you plan to work on.

For stage 1, you want to ensure things fit into the cache. A 100k exponent fits nicely into a CPU core's L2 (or even L1) cache and it does not react very well to parallelization. So 1 core per worker is a good choice. Larger exponents like 10M will barely fit into the L3 cache. So it's a better idea to have just 1 worker, since multiple workers will just fight over the L3 cache (a.k.a. cache thrashing).

For stage 2, I optimize my setup for RAM utilization, as mprime 30.8 is VERY memory hungry: Some workers are only doing stage 1 work (which uses almost no memory) while some workers only do stage 2 work. This way the stage 2 workers will constantly utilize the RAM, i.e. the RAM never goes unused.

[Mark Rose]

Originally Posted by preda

32GB is a bit low RAM for efficient wavefront P-1 second-stage. Maybe you could merge RAM to create systems with 128GB (or more).

No, unfortunately. The Skylake systems have two slots with 16 GB DIMMs. The Haswell systems have four slots with 8 GB DIMMs.

[Mark Rose]

Originally Posted by nordi

One key parameter is the exponents your are working on. Setup for 100k exponents is different than for 10M exponents, but you didn't mention what you plan to work on.

For stage 1, you want to ensure things fit into the cache. A 100k exponent fits nicely into a CPU core's L2 (or even L1) cache and it does not react very well to parallelization. So 1 core per worker is a good choice. Larger exponents like 10M will barely fit into the L3 cache. So it's a better idea to have just 1 worker, since multiple workers will just fight over the L3 cache (a.k.a. cache thrashing).

For stage 2, I optimize my setup for RAM utilization, as mprime 30.8 is VERY memory hungry: Some workers are only doing stage 1 work (which uses almost no memory) while some workers only do stage 2 work. This way the stage 2 workers will constantly utilize the RAM, i.e. the RAM never goes unused.

I'll probably run a bunch of P-1 to eliminate DC candidates to start. My Skiylake systems have 6 MB of L3. My Haswell systems have and 8 MB of L3 cache.

[kriesel]

Originally Posted by Mark Rose

I'll probably run a bunch of P-1 to eliminate DC candidates to start. My Skiylake systems have 6 MB of L3. My Haswell systems have and 8 MB of L3 cache.

Welcome to the party. There's plenty to do. It's now at ~85.5M and there are a lot of them.
Try one worker and two workers, and use whatever gives better aggregate throughput. Four workers would either leave 3 stage 1 oversaturating one stage 2 at a time, or divide the memory into 2 stage two 15 GiB workers which is below George's threshold of 24 GiB.

[mrh]

Originally Posted by kriesel

Welcome to the party. There's plenty to do. It's now at ~85.5M and there are a lot of them.
Try one worker and two workers, and use whatever gives better aggregate throughput. Four workers would either leave 3 stage 1 oversaturating one stage 2 at a time, or divide the memory into 2 stage two 15 GiB workers which is below George's threshold of 24 GiB.

Does it make sense to make a work type for this, rather than coordinating via the forum? Or is that too much work?

[kriesel]

Originally Posted by mrh

is that too much work?

I trust George to have done the sensible thing. A forum thread is a very quick way to get going. We've already churned through over 23m of exponent value in under 3 months. There's ~30M remaining, to catch up to the first test wavefront.


Edit: a quick start was useful to the project, because with the recent improvement in P-1 stage 2 efficiency and resultant factor productivity increase, some DC that was soon to be started was avoidable if we acted fast.
Perhaps long term it would make sense to implement a special high memory and suitable software P-1 work type. I think new users and old not adjusting upward from the prime95 conservative memory limit default, and limited-memory systems, will be with us for a long time.
George may choose to program other enhancements first, and is currently still working on the v30.9 ECM enhancement, optimizations for new chip designs, and doing occasional database maintenance or queries, etc. I think there is more to do with handling the disparate performance and economy cores of recent hardware.
There is no NUMA-awareness in prime95/mprime. Mmff could use extension to higher bit capable mm127 kernels. There is no Google TPU compatible TF code for GIMPS. Unfortunately there is only one of him.

[mrh]

Originally Posted by kriesel

I trust George to have done the sensible thing. A forum thread is a very quick way to get going. We've already churned through over 23m of exponent value in under 3 months. There's ~30M remaining, to catch up to the first test wavefront.

Oh, that makes sense, thanks.