How to use prime95 for stage 1 & GMP-ECM for stage 2

[Madpoo]

Originally Posted by Madpoo

This may sound dumb of me but could one reason Prime95 does stage 2 slower than gmp-ecm be due to it's low cap on memory? I may test this theory by comparing stage 2 runs on P95 with gmp-ecm and the "-maxmem 250" option. I'm also curious to try adding "MaximumBitArraySize=2000" to my prime.txt and see what happens. Would it use more RAM in stage 2? Who knows...stay tuned.

Well, interesting...

gmp-ecm with "-maxmem 250" performs horribly.

I fed it one of the same stage 1 outputs from Prime95 that I used before... B1=29e8 and autoselected B2. It finished up stage 2 in I think 128 minutes when it could gobble up as much RAM as it wanted.

With only being able to use 250 MB, it chose a much smaller B2 (80847864213490). It's been going now for several hours (5 or 6? I lost track) and it's still going.

On the other hand, with Prime95 I didn't see any difference in memory usage for stage 2 when I set that MaximumBitArraySize=2000. With B1=800000000 and B2=80000000000 it still does stage 1 in a reasonable time but then stage 2 didn't complete any faster than it had before and still only uses 207 MB of RAM.

Just for fun I tried doing a stage 2 with B1=29e8 and B2=425327623620922 which is what I have three threads doing right now (and using 37 GB per thread with gmp-ecm). I'm still waiting to see just how long stage 2 will take with that "-B2scale 4" option. Definitely longer than the 128 minutes from before but we'll see. I need to refresh my memory on how long the stage 1's were taking in Prime95 on different machines, see if they're closer now. If memory serves, it was 5 hour, 45 minutes on a similar system so if this stage 2 finishes pretty soon it'll be closer to that.

Umm... let's just say that stage 1 took as long as it has before when I was doing just stage 1 and the ecm hook. Stage 2 goes VERY slow. I think it was estimating completion of that single curve in somewhere around October. I didn't bother. It did start using a lot more memory at those bounds than I've ever seen it use with smaller ones... 18.8 GB for a single worker. Still terribly terribly slow though.

[Madpoo]

Originally Posted by VBCurtis

I mean, I appreciate that with perfect software and no memory constraint you are correct, but we do not have such software at our disposal. So, why would we choose your mathematically correct but in practice (with current software and memory systems) less efficient ratio?

Everything else considered, it may also bear noting that trying to optimize the ratio not just between stage 1/stage 2 but also between 2 totally different programs is going to cause confusion, ultimately. I just don't feel like it's totally apples to apples anyway.

Not to mention the "human factor" introducing inefficiency by having to digitally shuffle files around, launching different programs, etc. Heck, you're going to lose however much time just doing that.

Times like this I wish I'd spent a little more time in my programming classes so I could just hack the best parts of the ECM code into one or the other. LOL

[axn]

Originally Posted by Madpoo

Times like this I wish I'd spent a little more time in my programming classes so I could just hack the best parts of the ECM code into one or the other. LOL

There is (or at least there used to be) a provision in GMP-ECM to incorporate gwnum libraries to speed up stage 1 for Mersenne numbers. if you do "ecm -princonfig" you can see reference to GWNUM_VERSION. Since then, gwnum's scope has been expanded to support the general form (k*b^n+c)/f. So I guess it is time to revisit this integration.

[ATH]

Anyone can compile GMPECM with the --with-gwnum feature? I cannot get it to work.

[R.D. Silverman]

Originally Posted by VBCurtis

If 0.4K was determined by experiment on that implementation, why wouldn't we determine experimentally the correct ratio for the current (non-perfect implementation) GMP-ECM?

Because there is no such thing as the "correct" ratio. It will be different for different computers.
It will be highly dependent on aggregate DRAM size, L2/L3 cache size, memory latency,
memory bandwidth, etc.

The optimal thing to do for ANY computer is to measure how long step 2 takes relative
to step 1 and select the B2/B1 ratio so that GMP-ECM spends as much time in step 2
as it does in step 1, FOR THAT COMPUTER. This ratio will be DIFFERENT
for different computers.

This must be done on a case-by-case basis.

I have done empirical studies with GMP-ECM and composites around 200 digits, and found that stage 2 time of 40 to 50% of stage 1 time is quite a bit more efficient than stage 2 time equal to stage 1 time.

Oh really? Explain how you determined that your selection of parameters gave the largest possible
probability of success per unit time spent. How did you do this analysis?

If you agree this is possible, please stop telling us to choose B2 to make stage 2 time equal to stage 1 time.

Read what I wrote above.

Oh, and as you ask me to stop telling you certain things, I can ask you to stop prattling
about optimal parameter selection. You clearly do not understand it.

[R.D. Silverman]

Originally Posted by Madpoo

Times like this I wish I'd spent a little more time in my programming classes so I could just hack the best parts of the ECM code into one or the other. LOL

Wrong class. You should have spent more time in your MATH classes.

[Madpoo]

Originally Posted by R.D. Silverman

Wrong class. You should have spent more time in your MATH classes.

Nah, programming. Just in the sense that both gmp-ecm and prime95 have source code, so in theory it would be possible to take the relevant libraries from one or the other and fit it in.

[Madpoo]

Originally Posted by Madpoo

gmp-ecm with "-maxmem 250" performs horribly.

So horribly in fact, I just called quits on it after it had been running about a day. There's no real sense of the progress of gmp-ecm so I have no clue how far along it was, but it proved the point that you're really hobbling stage 2 by limiting how much RAM it can use. Don't do it. Feed that beast with yummy RAM chips.

Originally Posted by Madpoo

Just for fun I tried doing a stage 2 with B1=29e8 and B2=425327623620922 which is what I have three threads doing right now (and using 37 GB per thread with gmp-ecm). I'm still waiting to see just how long stage 2 will take with that "-B2scale 4" option. Definitely longer than the 128 minutes from before but we'll see.

It ended up taking 4.55 hours. (reminder that stage 1 on Prime95 took 5.75 hours).

I won't wade into the perfect ratio "debate", but suffice to say that "-B2scale 4" with ample memory available did get it closer to "whatever", compared to the 128 minutes stage 2 took previously.

[Dubslow]

Originally Posted by Madpoo

Nah, programming. Just in the sense that both gmp-ecm and prime95 have source code, so in theory it would be possible to take the relevant libraries from one or the other and fit it in.

No you really do mean math. No amount of programming experience and knowledge can tell you which pieces fit where without understanding what the pieces are doing.

[lorgix]

Originally Posted by Madpoo

So horribly in fact, I just called quits on it after it had been running about a day. There's no real sense of the progress of gmp-ecm so I have no clue how far along it was, but it proved the point that you're really hobbling stage 2 by limiting how much RAM it can use. Don't do it. Feed that beast with yummy RAM chips.




It ended up taking 4.55 hours. (reminder that stage 1 on Prime95 took 5.75 hours).

I won't wade into the perfect ratio "debate", but suffice to say that "-B2scale 4" with ample memory available did get it closer to "whatever", compared to the 128 minutes stage 2 took previously.

You limited available RAM very strongly. More realistically people will be using a k in the area of 4~20 or so.
If you tell it to use something like half or a quarter of what it "wants" you wont see as dramatic results as when limiting to 250.

If I were you I'd try B2scale 3 next. According to the estimated time to find a factor (given by -v), that should be close to optimal. In practice that is. (Not sure why RDS seem to disagree)

[Madpoo]

Originally Posted by Dubslow

No you really do mean math. No amount of programming experience and knowledge can tell you which pieces fit where without understanding what the pieces are doing.

I'll step back a bit and say that I don't think I'd have problems figuring out the math part. It's a little rusty (a lot rusty?) but nevertheless...

Sadly for me, my college years involved Pascal, and since it was an electrical degree, I did some work in assembly on specific devices. Beyond that, my knowledge of C/C++/C# are what I'd best call "anecdotal". I have a passing familiarity with it and that's about it. I've been known to fire up Visual Studio and debug some things or modify some things where necessary (and I'm sure the devs I work with shudder in fear), but that's about it.