Small inconsistencies between mersenne.org and mersenne.ca factor databases

[GP2]

And here are the "new" factors for the range 20M to 500M:

Unlike all the prior exponents listed, 69577009 does not have any other known factors.

23209931,5249614763059849937,
26734819,115799942904981848081,
29990993,88809201997663095449,
30104329,11605899234298153241,
30238081,4217276702733398191,
31076581,23275543395572279657,
39995863,112209065709864656887,
40022053,4868324859045899161,
40917581,4543088939143018937,
45144431,5812435048411062353,
45180937,6344081429247326351,
69577009,359294707572838294767689,
75657689,8940306667834886417,
92900947,854260774950962417527,
102050951,4172692574528725711,
152843819,7476581080895128777,
182084179,4427470744878757447,
207137831,4588312505565352841,
214952987,4185170884206617033,
224357201,7333864736516266799,
235914361,4199637616570148177,
250862839,12806891159981142767,
299602489,8653205878543833433,
329069747,4816427236047026791,
341348477,16678532903281003201,
347655431,11280617285189604839,
380312783,5428276280315648791,
408030461,4504517823486998729,
422391611,4046525877269906143,
434669639,4714823392851910241,
453286103,4986934345909358041,
478687763,4700805262022733001,

James Heinrich's post mentioned the bogus large composite factors which have already been removed at mersenne.ca:

48152443,2113910962084939992015744887285933617190716851808148277973015704520247,
63323077,68205072662349548620633421760422165673044571137810937540436645541918441353903,
71099111,1775710684378731227619986484940543071180369205598911746485081,
72051011,51697592340892129034646687894365125162666678844373701616107849,
77999869,1815615329758341197381057535338581863482459211614518833203801,

[GP2]

The factor

422391611,4046525877269906143,

in the previous post is actually already in mersenne.org. It was rediscovered on June 2, which was after the beginning-of-month cutoff date for the data comparison. The original discovery recorded at mersenne.ca was 2015-02-16.

That makes you wonder if the original list of inconsistencies was longer, and it gradually got reduced through rediscoveries.

Here is the final set of "new" factors in the range from 500M to 1000M (but nothing was found in the 500M's):

649603081,4472616496720693879,
660634967,6567519593286867767,
665453653,4887142138226077313,
686309399,43721950469217363689,
686458511,4737085242159976921,
695354567,4268469818898991367,
726064763,34153055801363324180057,
736098431,135916687847990950886081,
739955179,25469172817494972521,
785541607,5638659929753122313,
789858929,29263462361809288553,
791903171,5169043843527833999,
832608499,4370027229364854089,
858453133,5668759900647139199,
875944451,4771042626811635983,
899990243,11361555126783141001,
930914249,5590695223404705143,
967554767,7426722474291085481,
987859777,5322803294511667313,
992070601,4993735061668012073,

These factors occur twice in Detailed Reports -- Factors Found):

903045349,4698698083859011327,
903046847,6409898135233154489,
903047281,5338409802383497759,
903047407,2229309709723286999,
903047723,3129247480713682831,
903048053,11654468353902436343,
903049057,4151476974203742103,
903049099,8262430436104155953,
903049739,10509312940426305311,
903050011,10382857575219788137,
903050237,5966672212578553081,
903050327,10577728097800693369,
903050887,10408369083482933687,
903051263,4531360271042159239,
903051293,4878578908134663527,
903052343,2165181878079580417,
903053927,12736331282594613913,
903054049,18408987089320792177,
903054377,11473458517515214343,
903055183,6709762091121610519,
903055211,18037548908009061359,
903055919,3474986262571234487,
903056437,6812054490130278017,
903057787,4105727970657225607,
903058073,7525267323922618727,
903061057,5118622532695213681,
903061349,1808442662605581001,
903061451,4890945808729667633,
903061477,9660000345414685489,
903061771,3943540281398449463,
903062137,4142750729715696551,
903122963,6020978707791477137,
903123607,2724889658144614999,
903124421,3117856583118207361,
903126281,6252955779731825441,
903126281,9659963773728403129,
903126517,2388597897120274247,
903126827,1460815310219261231,
903127003,5531410098517767487,
903127903,3048046436573347399,
903128201,1568929402049438711,
903128243,3868939830372187663,
903128563,13958719956089470561,
903129457,1646246063776639313,
903130127,1728284880289823953,
903130271,2527301196458112577,
903130391,2711679873393046727,
903131023,4880200255015541839,
903131101,2171104494600840497,
903131809,1297474126073044481,
903132169,6128282447127504401,
903132721,3312271180009187351,
903133241,2077804689758838401,
903133331,5476057064120870737,
903133379,1356337304814786361,
903134689,10524355235295058567,
903135349,9484902665130954377,
903135781,10246545969186901001,
903136019,2067730488088736047,
903136939,13671895325436184033,
903137057,2977070372929745359,
903137639,12260997679860543191,
903137743,2082303376401176759,
903137779,2043532309078058881,
903138253,11309495854709703719,
903138541,2061303349077681473,
903139247,14797629691635375847,
903141937,5809710434414442137,
903142013,16917974720285133023,
903143167,16844977837343263463,
903144317,16816090767721672247,
903144667,1621079602079163983,
903145081,6226850409366871799,
903145277,1993492157032549247,

Finally, here is the opposite situation: three factors that were known to mersenne.org but were not in mersenne.ca. I think these are the only ones:

332288357,6511596732777806863,
332293097,7424125886926591033,
332296499,8355693478871973049,

PS,
If anyone else is working with the Mersenne.ca SQL dump, note that the table adds an extra field for 82M and higher ("date_found", second to last).

[GP2]

Originally Posted by James Heinrich

The composite factor (which was found via P-1) for M499211 is just over 200 bits, which is actually a cutoff point in the factoring code beyond which it was supposed to fall back to a probable-prime test rather than attempting to fully factor the number, but the PRP section was commented-out for whatever reason. I have re-enabled it, and removed the composite factor for M48152443, M63323077, M71099111, M72051011, M77999869.

How does that PRP code work? It shouldn't reject composite factors outright, because often these are not duplications of known prime factors, but rather constitute a new discovery of multiple prime factors that have to be disentangled from the composite factor that was reported. Maybe it should simply flag each new large composite factor for asynchronous offline factoring if none of the existing known prime factors divide it.

[GP2]

Originally Posted by GP2

Unlike all the prior exponents listed, 69577009 does not have any other known factors.

69577009,359294707572838294767689,

I forgot to mention it earlier, but an unnecessary LL test was done on this exponent. The LL result was reported on 2015-03-02, but the factor was found on 2014-03-11.

[James Heinrich]

Originally Posted by GP2

factors for non-prime Mersenne exponents (M100, M77777)

I know several years ago someone kept trying to submit those kind of factors, they must have got into the database before I had a check for prime exponent. I thought I'd purged all that spurious data, but I guess not. I've removed them now, thanks.

Originally Posted by GP2

three factors that were known to mersenne.org but were not in mersenne.ca

Thanks, imported.

Originally Posted by GP2

If anyone else is working with the Mersenne.ca SQL dump, note that the table adds an extra field for 82M and higher ("date_found", second to last).

Nobody else is, but I'm curious what you mean; the table structure should be consistent from 0M to 4294M

Originally Posted by GP2

How does that PRP code work?

It's part of the check-if-submitted-factor-is-prime function. Below 200 bits the submitted factor is factored completely, above that it's run through gmp_prob_prime. In any case it's accepted, but if it's probably composite then I'm alerted so I can look into it.

[bloodIce]

Originally Posted by James Heinrich

I know several years ago someone kept trying to submit those kind of factors, they must have got into the database before I had a check for prime exponent. I thought I'd purged all that spurious data, but I guess not. I've removed them now, thanks.

These are my fault, I am sorry... About 5 years ago, I just tried to submit factors, to non-prime exponents to James' database and the database accepted them. Later, I agreed in a conversation with James not to do it any more , and James removed them. However, in some recovery from a backup, these were resurrected. There is no malicious activity (at least not intended) or a bug, it is an old story.

[GP2]

Originally Posted by James Heinrich

Nobody else is, but I'm curious what you mean; the table structure should be consistent from 0M to 4294M

From 0M to 810M, in 10M increments

CREATE TABLE `known_factors_000` (
  `exponent`,
  `factor`,
  `factordigits`,
  `factorbits`,
  `class4620`,
  `timestamp_found`,
  `k`
)

From 820M to at least 1000M:

CREATE TABLE `known_factors_082` (
  `exponent`,
  `factor`,
  `factordigits`,
  `factorbits`,
  `class4620`,
  `timestamp_found`,
  `date_found` int(11) DEFAULT NULL,
  `k`
)

[James Heinrich]

Well, what do you know, thanks
From 820M to 3770M I apparently missed removing that old field. I'm running through and dropping that field from the affected tables.

[GP2]

Just for completeness, here are the largest factors of fully-factored exponents that are stored in mersenne.ca (at the bottom of this post), which constitute the final inconsistency between mersenne.org and mersenne.ca

These seem a bit haphazardly chosen. For instance, the list is complete up to 97, but then it omits:
2^101−1 = smaller factor * 7432339208719
2^103−1 = smaller factor * 2550183799
2^109−1 = smaller factor * 745988807
etc.
while at the same time including some considerably larger ones.

Overall the list is very incomplete and inconsistently selected, since there are just over 270 fully-factored and about 30 probably-fully-factored Mersenne numbers with prime exponents. It would be impractical to include them all since they start getting humungous and are easily calculated from scratch from the complete list of smaller primes. Maybe mersenne.org has the right idea in omitting these altogether.

11,89,
23,178481,
29,2089,
37,616318177,
41,164511353,
43,2099863,
47,13264529,
53,20394401,
59,3203431780337,
67,761838257287,
71,212885833,
73,9361973132609,
79,1113491139767,
83,57912614113275649087721,
97,13842607235828485645766393,
191,87274497124602996457,
223,596242599987116128415063,
239,7110008717824458123105014279253754096863768062879,
251,12070396178249893039969681,
281,48009215293052652841860443273079338843737271906291675944391068955229998769420319,
359,100361196281293745682520861860411315001,
373,752440346497356983142327449546457327748644897934114291899411428982990336039662496766303354959577078458241,
383,240522700235167893496900256599634325263,
397,6597485910270326519900042655193,
431,1807482391092819529831423005040763105191863029850140579776353298087457,
491,14797326616665978116353515926860025681383,
809,8229761617709728309303010031634283091737016329953222983325661319412184753194569346813172621804415601791969836957...

[James Heinrich]

Originally Posted by GP2

largest factors of fully-factored exponents that are stored in mersenne.ca
These seem a bit haphazardly chosen.
Maybe mersenne.org has the right idea in omitting these altogether.

I've been meaning to get rid of those for a while. I finally have now, thanks for the push.

[Madpoo]

Originally Posted by GP2

Here's what I have so far (only looked at < 10M for now):...

James, George and I worked through the factors and cleared up some of the inconsistencies. Specifically, 81 factors that James had and Primenet was missing were reconciled. 80 of those were for exponents where Primenet already had another factor and only one was for a newly factored exponent.

We also found 77 cases in Primenet where the same factor for an exponent showed up twice. They were from the same user and were probably the result of a race condition when a user was submitting things weird.

George found the instances of spaces on the end of factors. They're stored as strings because of their length. Now, there are checks on the input in general (you know, to avoid SQL injection) and all submitted factors are tested to make sure they're valid, but I guess when passing to the verification it's just ignoring the space. Easy enough to fix. George already took care of that.