More missed factors

[LaurV]

He he...

[lycorn]

Originally Posted by LaurV

I long time suspected Nooe of foul play, see former posts, nobody jumps to the first place in ECM tops in so short time...

May be (s)he´s testing very small exponents and using GMP-ECM for Stage 2.

[Madpoo]

Originally Posted by LaurV

[edit, otoh, I long time suspected Nooe of foul play, see former posts, nobody jumps to the first place in ECM tops in so short time..., but George said he knows the person and it is a honest user, see the discussion about the 383....83 exponent]

Yeah, I glanced at that thread too, especially since I'm doing a triple check of that 383M exponent. It also did not escape my attention that several of the missed factors were originally TF'd by NOOE. I don't suggest anything untoward, but maybe more along the lines of problems on the system doing the work.

I guess for me, it didn't matter if the user is trustworthy or not since I'm just aiming to do triple-checks of all the self-verified exponents anyway. If I started making exceptions because user XYZ was trustworthy, that may be true, but not scientifically thorough. If someone looked back years from now and wondered why we let people self verify, they'd have no first-hand knowledge of the people involved so it doesn't hold up over time.

That's why I even check your self verified work. :) In fact, since you mentioned you'd been double-checking your old numbers, I thought I'd save you some time and I dug up your last few v4 first time checks and went ahead and did a double-check before you got to them.

You have 169 unverified checks since the v5 database went live, starting around exponent 40M. But I figure you were probably just doing your own DC's lower down near the current DC assignments. I'd just encourage you to let those get assigned as DC's normally, otherwise some stickler like me will probably just triple check 'em anyway. LOL

A lot of the triple checks are for the heavy hitters like you, Curtis, TheJudger, etc. and though I trust you folks, it's really not personal.

And FYI for TheJudger, I've noticed that your self doublechecks follow a pattern... when you check in a result that didn't match someone's first time run, you run your own triple-check. It says a lot that you cared enough to make sure your own machine had the proper result in those cases.

[TheJudger]

Originally Posted by Madpoo

And FYI for TheJudger, I've noticed that your self doublechecks follow a pattern... when you check in a result that didn't match someone's first time run, you run your own triple-check. It says a lot that you cared enough to make sure your own machine had the proper result in those cases.

I don't feel comfortable with "selfchecking", primenet should avoid this whenever possible. Currently (at least in the last 2 years) all of my LL runs (first time and doublecheck) are done fully automatically with prime95/mprime using builtin primenet communication.
Anyway, machine errors *should* be non-reproduceable but I still feel uncomfortable with "selfchecking"!

Oliver

[Madpoo]

Originally Posted by TheJudger

Currently (at least in the last 2 years) all of my LL runs (first time and doublecheck) are done fully automatically with prime95/mprime using builtin primenet communication.

Cool... yeah, these are mostly exponents that were first-time checked under the old v4, so I guess pre-2009, and then your double-check happened in 2009-2011'ish. It looked like you probably noticed your residue didn't match the first run so you ran a triple check and it came back in an additional couple weeks.

I have no doubt your runs were solid (especially now that they've been independently run).

I think it was ATH who did some investigation a while back (is that right?) and looked for other suspicious things, like the matching runs being done within a day or two of each other, whether by the same person or not.

After looking at the self-verified stuff so far, I'd have to say that's kind of common, especially with some of the larger exponents (60M +) that were done in recent years. Probably explained by people running LL on 2 machines and comparing results along the way.

For the most part I've been satisfied that every case of a self-verified check was on the up-and-up. There were a few that made me wonder, and I'm sorry but I don't recall specifics, but things like a user who hadn't done many tests, it was a smallish exponent (20M range) and the first/second tests were checked in pretty close to each other. Oh well, my runs matched so all's well that ends well.

[bloodIce]

Originally Posted by Madpoo

Overall, on first glance, it seems like maybe that user "KYOJI_KAMEI" might have had some issues since there seemed to be an unusual amount of false negatives in there (12 of them). Beyond that, I couldn't say whether the mfaktc results are relevant or not until I do that extra check.

EDIT: Indeed, after double-checking who actually submitted the factor eventually, user "KYOJI_KAMEI" missed all 12 and someone else found them.

With a help of a spider I have built a list of KYOJI_KAMEI's results for a given period, for which I suspected to find his "hardware error". Then I systematically double checked all and found couple of factors. If he was not targeted specifically, his misses would be less. Nevertheless, most of his "hardware errors" are corrected. Someone could triple check though . I can dig in some of my logs the time period in question if someone is interested.

Originally Posted by Madpoo

"Never Odd Or Even" has 14 where "no factor found by TF" but then he later found a factor by ECM. The other 17 were "no factor found by TF" followed by a "factor found by TF"

NOOE is another favourite aim of mine. Personally I think that he (or someone else) uploaded a very big bunch of wrong lines on one day (2014-03-08). NOOE went through most of these and latter found some of the missed factors. The rest of the missed factors of NOOE were found by other users (including myself). I would like to mention that Mark Rose, did tremendous amount of work to double check a list of NOOE results and there were no misses discovered (thanks again Mark!). I think that NOOE corrected most of the problems himself. I continue to double check NOOE by ECM in an effort which will take years. The famous date 2014-03-08 would be ECMed at the first bonds, one day ...

[Madpoo]

Originally Posted by bloodIce

If he was not targeted specifically, his misses would be less.

I think that goes to show that it's really hard to find TF problems because usually they're not being specifically checked... the only way people find out is if ECM or P-1 finds a factor that *should* have been found in TF but wasn't.

Plus, you'd have to build up a large enough set of those for one user to really start to say "hey, maybe this user had a hardware problem or something" and then check all of their previous work.

I have a feeling a lot of missed factors (well, don't ask me to define "a lot") are probably out there. Who knows...maybe down the road someone will fire up their Nvidia MegaTitan LXIV and re-TF all exponents below 100M to 74 bits on their lunch break.

[VBCurtis]

Originally Posted by Madpoo

I think that goes to show that it's really hard to find TF problems because usually they're not being specifically checked... the only way people find out is if ECM or P-1 finds a factor that *should* have been found in TF but wasn't.

Plus, you'd have to build up a large enough set of those for one user to really start to say "hey, maybe this user had a hardware problem or something" and then check all of their previous work.

I have a feeling a lot of missed factors (well, don't ask me to define "a lot") are probably out there. Who knows...maybe down the road someone will fire up their Nvidia MegaTitan LXIV and re-TF all exponents below 100M to 74 bits on their lunch break.

This observation leads me to think that rather than double-check a bunch of TF ranges, it may be more efficient to run ECM with B1=50k as a way to discover missed factors. The no-GPU folks who follow your work may be particularly interested in doing, say, a dozen curves at 50k on each suspicious candidate. This has a chance to find factors around 25 digits, and is highly likely to find 21-digit or smaller factors. A full T25 is overkill, but the combined chance of larger factor plus finding missed small factors may change the effective efficiency of ECM work.

[bloodIce]

Originally Posted by VBCurtis

This observation leads me to think that rather than double-check a bunch of TF ranges, it may be more efficient to run ECM with B1=50k as a way to discover missed factors.

This is very good idea! However, after certain borders ECM will be substantially expensive method for the purpose of a double check. Not to mention, that where ECM is the most efficient, there is usually Pminus1 effort done or in progress. Personally, I am slowly doing ECM in 2M range (including NOOE's), I know that VicktordeHolland is having similar activity in 1.5-2M range. And I agree with VBCurtis, it is rewarding effort in 2M (I have found many factors).

But what about 20M? There an ECM is an overkill. Most of the discussed misses are at 50M-80M. In these "higher" ranges TF double check and/or Pminus1 are the only alternative. So every method should be used in its own range. However, more ECM efforts are welcomed .

[bloodIce]

Originally Posted by Madpoo

I think that goes to show that it's really hard to find TF problems because usually they're not being specifically checked... the only way people find out is if ECM or P-1 finds a factor that *should* have been found in TF but wasn't.

That is where the work of TJAOI is indispensable (no matter that (s)he overloads your server). Actually, his work pointed us to KYOJI_KAME, sannerud.com and 1997rj7. Once you see a user/machine with misses, you start to check. The beauty of TJAOI's work is that it is an alternative approach, completely unbiased on a range and so far it seems systematic. Unfortunately, after several more bitlevels, his effort will be less productive (not to mention that there is where the GPUs kicked in). He will continue to find misses though if they exist, so it would be nice if someone makes a special QUERY in his result log to see which of his factors are actual first for an exponent. Madpoo, is it possible to have such list of TJAOI's firsts?

[Madpoo]

Originally Posted by bloodIce

...But what about 20M? There an ECM is an overkill. Most of the discussed misses are at 50M-80M. In these "higher" ranges TF double check and/or Pminus1 are the only alternative. So every method should be used in its own range. However, more ECM efforts are welcomed .

Just for fun I took one of the exponents that "KYOJI_KAME" has cleared in a certain range and I'm doing 12 ECM curves and also P-1 to get an appreciation of the time involved. It's a larger exponent, 47590559.

12 curves with b1=50K will take it about a week. P-1 with B1=50K is about 4 hours (each running on one core of a X5690 at 3.47 GHz with as much memory as it could need).

I have 3 other cores on this box doing curves of that pesky M1277. In fact I have a few machines that I can't dedicate *all* the cores to doing work, but I have 3 out of the 12 cores on each box doing 150 curves each (12 assignments total) of M1277. I figure it's a fun way to put some extra resources to work there, see if I can't find a factor for that fella if I throw enough curves at it over time. It's already had a lot done on it so maybe it's just a matter of time.