Best use of large capacitor server

[chalsall]

Originally Posted by airsquirrels

Multi-month jobs are not a problem, this is a dedicated number theory research cluster.

If I may ask you, just what the heck do you do for a living?

You are currently single handedly staying ahead of the GIMPS "churners" in the DCTF domain!

[VBCurtis]

Originally Posted by ATH

Off topic: 768 bits is still the record for biggest GNFS? and 1061 bits for SNFS?

The CADO group did a bunch of 2^n-1 factorizations in the 1100s, all at once. Some of the relation-gathering work was re-used in multiple factorizations. See https://eprint.iacr.org/2014/653.pdf for details.

[airsquirrels]

Originally Posted by chalsall

If I may ask you, just what the heck do you do for a living?

You are currently single handedly staying ahead of the GIMPS "churners" in the DCTF domain!

I know - That was on purpose :) I would like to see DCTF get taken care of so we can move all our resources to the LL domain so we can accelerate this search for #49, though as always everyone's passion and use of their resources is their own. I'm hoping to turn on another 10,000GhzDay/Day or so of GPU power over the weekend. Once I get some more time I'm hoping to contribute on the Math and kernel side - the BitTorrent crowd had a bit more motivation/monetary support in creating highly tuned GCN-ISA kernels than those of us purely interested in research have had.

To answer your question, my company does education and business focused wireless screen mirroring/sharing/collaboration software, but my background is encryption/security. Originally credit card processing followed by a bit of time in the medical industry (glad to be out). The number theory side of that is a passion stemming from that work, and we decided to kickoff an R&D investment by building a dedicated research cluster. It doesn't hurt that AMD's Fury X cards kick out a solid 1000GhzDay/Day for ~650 USD and 300 Watts of power. Two racks and a lot of liquid cooling later, we have a pretty good workhorse. Unfortunately I also believe I have contributed to AMD's supply crunch as I have been trying to get every card I can find.

I'm just here to help with everyone else, if there are better places to put resources or immediate needs just let me know!

[Mark Rose]

Originally Posted by airsquirrels

I'm hoping to turn on another 10,000GhzDay/Day or so of GPU power over the weekend.

Yikes! I guess I won't be #1 for long lol

I'm just here to help with everyone else, if there are better places to put resources or immediate needs just let me know!

I noticed you're taking many exponents above the recommended level. Is there a reason why?

[airsquirrels]

Originally Posted by Mark Rose

I noticed you're taking many exponents above the recommended level. Is there a reason why?

I wanted to find factors :) If it wasn't for that pesky effort doubling with every bit I would probably not stop till I succeeded in factoring every exponent I looked at.

I took the first couple groups of assignments up to higher levels to get an idea of any GPU performance changes on longer jobs/different kernels and to verify my hit-rates. All of the newly taken assignments since yesterday are at recommended levels.

On the topic in this thread, it does seem like I am in a fairly unique position with this much horsepower in one place/machine. One area I intend to explore is looking at working with multiple (very many) exponents at once. In the RSA-sized world I've done some work with pretty massive GCD trees along those lines, I'm not sure if there is something equivalent that would apply here. I need to brush up on the math involved, but at first glance the 2kp+1 aspect makes it seem like factors would be pretty rarely shared (Only when k = another p or a multiple of other p's).

I'm sure all of this has been thought about and is spelled out in a paper or somewhere in this forum. I will continue researching, but I do intend to at least setup the large servers CPUs to look at the other types of work suggested in this thread.

[Mark Rose]

Originally Posted by airsquirrels

On the topic in this thread, it does seem like I am in a fairly unique position with this much horsepower in one place/machine. One area I intend to explore is looking at working with multiple (very many) exponents at once. In the RSA-sized world I've done some work with pretty massive GCD trees along those lines, I'm not sure if there is something equivalent that would apply here. I need to brush up on the math involved, but at first glance the 2kp+1 aspect makes it seem like factors would be pretty rarely shared (Only when k = another p or a multiple of other p's).

I'm sure all of this has been thought about and is spelled out in a paper or somewhere in this forum. I will continue researching, but I do intend to at least setup the large servers CPUs to look at the other types of work suggested in this thread.

I found reading the source code of mfaktc insightful. The 0.21 version is far cleaner if you looked at older versions previously. I'm not sure what the state of mfakto's code as I don't use it, but I understand it's a fork that implements the same general algorithm.

[LaurV]

Originally Posted by airsquirrels

but at first glance the 2kp+1 aspect makes it seem like factors would be pretty rarely shared (Only when k = another p or a multiple of other p's).

They are never shared, just to point out. There is a theorem about it, which you can prove very simple, assuming some odd prime x divides 2^p-1 and 2^q-1, it will divide the difference, i.e 2^(p-q)-1. Repeat (already recognize Euclid's algorithm at the exponents?) then you reach the fact that x divides 2^gcd(p,q)-1 which is impossible if (p,q)=1 (here they are both prime, but this is not required, only they be prime to each other is enough).

Related to "going over the recommended TF level", if you use Fury X, you can safely go 1 or 2 bits over the recommended level, as their ratio LL to TF is very low (they are much better doing TF than other cards, but worse at doing LL than the same other cards). But please! don't go higher than that! We know that finding factors is fun, but you waste your time without bringing any benefit for the project. For example, assuming that 72 is the recommended level for some range of expos you are working with, in the same time you use to factor an exponent from 72 to 73, you could factor 2 other exponents to 72, and have 3 exponents TF-ed and ready for LL, instead of only one (even if that one is factored higher). In the same time you use to factor to (73 and then) 74, you could TF other six expos to 72, and have 7 (seven) expos ready for LL, instead of one. You will also find MORE factors in this way (that is true! the probability to find a factor of 73 or 72 bits is about the same as 72 bits, but you spend a double time). And you will help the project more, too.

[chalsall]

Originally Posted by airsquirrels

I'm just here to help with everyone else, if there are better places to put resources or immediate needs just let me know!

Wow! You've got some serious kit! Thanks for helping out!

To answer your question, what you're doing is great -- many here would like to see DCTF "Die baby die!". On the other hand, we're currently /really/ tight "feeding" the P-1'ers at 75 bits. So, if you're so inclined, doing a bit of "What Makes Sense" or "Let GPU72 Decide" LLTF'ing would be much appreciated -- these options will give your machines candidates not yet P-1'ed.

But, again, entirely up to you.

[airsquirrels]

Originally Posted by LaurV

They are never shared, just to point out. There is a theorem about it, which you can prove very simple, assuming some odd prime x divides 2^p-1 and 2^q-1, it will divide the difference, i.e 2^(p-q)-1. Repeat (already recognize Euclid's algorithm at the exponents?) then you reach the fact that x divides 2^gcd(p,q)-1 which is impossible if (p,q)=1 (here they are both prime, but this is not required, only they be prime to each other is enough).

Thanks! I had a gut feeling that was the case but had not yet settled down to look into it. Of course as with most math, this raises just as many new questions and potential approaches as it eliminates...

Originally Posted by LaurV

Related to "going over the recommended TF level" ...

I totally agree regarding going over the recommended level, that was only temporary while I was fiddling with the hardware configuration.

Originally Posted by chalsall

...So, if you're so inclined, doing a bit of "What Makes Sense" or "Let GPU72 Decide" LLTF'ing would be much appreciated ....

I've put some LL to 75 work in queue for a few cards to help out (And immediately found a factor on the first assignment!), once I get the hardware situation settled this weekend I will certainly look more closely at the distribution of work. That said, if I can get under full steam (~24GhzDay/Day) three weeks of work should nearly clear the DCTF pile to current release levels. Of course that all depends on adequately addressing the power, cooling, and supporting infrastructure issues....

[chalsall]

Originally Posted by airsquirrels

I've put some LL to 75 work in queue for a few cards to help out (And immediately found a factor on the first assignment!), once I get the hardware situation settled this weekend I will certainly look more closely at the distribution of work.

Nicely nicely...

Originally Posted by airsquirrels

That said, if I can get under full steam (~24GhzDay/Day) three weeks of work should nearly clear the DCTF pile to current release levels. Of course that all depends on adequately addressing the power, cooling, and supporting infrastructure issues....

OMG! If you pull that off, you'll almost double our aggregate throughput!

One advantage of generating a large buffer of "optimally TF'ed DC candidates" is that that is where the "churners" live. Churner's refer to new uses who haven't yet proven their commitment to the project; often overclockers who use Prime95 / mprime to test the stability of their machines and don't read the "Only testing" language in the [G]UI.

One fear I've always had in the back of my mind is when the next MP is found and announced -- there is always a surge of new users who don't appreciate just how much work is involved. The last time this happened we had to release for LL'ing candidates not yet optimally TF'ed. Fortunately (?) most of these were never actually completed and were subsequently TF'ed appropriately.

But, around here balance is everything; eliminating DCTF would be cool, but we also have to "feed" the P-1'ers and LL'ers.

[airsquirrels]

I have noticed that a ton of CPU cycles in GIMPS get wasted on half completed assignments by users that abandon, even amongst some of my coworkers who I convinced to run prime95 on their machines... Until they stopped. In 2015 with fast and very present Internet wouldn't it be possible to actively checkpoint to PrimeNet so other users could pick up where an exponent was left off? Just wait till the residue for a specific iteration is small and upload a checkpoint :)