Small inconsistencies between mersenne.org and mersenne.ca factor databases

[GP2]

Yesterday I noticed that Mersenne.ca had two known factors for M100003 but Mersenne.org had only one. I discovered that by sheer accident, and only because 100003 is very close to a round number.

I manually reported the "new" factor at Mersenne.org, and it gave me 0.0023 GHz-days credit. Mersenne.ca reports that this factor was originally discovered 2014-12-06. It's not clear by whom.

I decided to check all the factors below 1M. I found the following differences:

* Very small fully-factored exponents < 1000, where Mersenne.ca sometimes lists the largest factors but Mersenne.org systematically omits them. For instance M11 = 23 * 89, and Mersenne.org omits the "89". The largest of these exponents is M809. This is not really an issue, it is just mentioned for completeness.
* M100003, already mentioned above
* M499211, where Mersenne.ca gives a sixth factor which is actually composite, it is the product of the first, second, fourth and fifth factors (C61 = P6 * P9 * P22 * P25)
* M954257, where Mersenne.ca had four known factors but Mersenne.org had only three. I manually reported the "new" factor at Mersenne.org and got 0.0283 GHz-days credit. Mersenne.ca reports this factor was originally discovered 2015-06-02. It's not clear by whom.

Some remarks:
* For manually-reported new factors of unknown origin, Mersenne.org assumes they were found by ECM at a very early stage, so very little credit was given and this didn't distort the stats.
* It would be good to figure out how the inconsistency arose. Maybe look at the log files for Mersenne.org to see if the "new" factors were originally reported at the time and somehow rejected, or perhaps never reported there, or perhaps were accidentally dropped from the database somehow. As far as I know, Mersenne.ca doesn't even allow reporting of factors under 1000M, they are supposed to be reported directly to Mersenne.org instead. Or maybe it was non-GIMPS participants who originally found those factors.
* It might be a good idea to systematically compare the two databases to look for more inconsistencies. However it's hard to harvest factor data from Mersenne.ca, except through very clumsy and slow screen-scraping. Perhaps a compressed text file could be made available for public download, or both sites could exchange data dumps. "Browse by range" doesn't work well, for example the range http://www.mersenne.ca/exponent/browse/100001/109999 gets truncated after 106621.
* In the same vein, Mersenne.org lets you get a listing of all factors found after a given date, so a private mirror of data can be kept up-to-date easily, but Mersenne.ca does not provide a way to do this.
* Mersenne.org only maintains data for factors of exponents below 1.000 billion, but Mersenne.ca records them for exponents below 2³² ≈ 4.29 billion. Maybe Mersenne.org could consider adopting the higher limit and incorporating the Mersenne.ca data, this would make that data more secure and accessible.

[S485122]

Originally Posted by GP2

* Very small fully-factored exponents < 1000, where Mersenne.ca sometimes lists the largest factors but Mersenne.org systematically omits them. For instance M11 = 23 * 89, and Mersenne.org omits the "89". The largest of these exponents is M809. This is not really an issue, it is just mentioned for completeness.

This is known and by design. See the bottom of the page Reports / Detailed Reports / Factors Found : "If a Mersenne number has been fully factored the largest factor is not listed."
I would prefer to have all factors or an indication that a Mersenne number is fully factored....
Jacob

[James Heinrich]

The composite factor (which was found via P-1) for M499211 is just over 200 bits, which is actually a cutoff point in the factoring code beyond which it was supposed to fall back to a probable-prime test rather than attempting to fully factor the number, but the PRP section was commented-out for whatever reason. I have re-enabled it, and removed the composite factor for M48152443, M63323077, M71099111, M72051011, M77999869.

To clarify, up until mid-2015 mersenne.ca accepted factors independently from mersenne.org and generally required people to submit results in two places. But for the last year or so mersenne.ca does not accept any results submissions below 1000M, that data is all pulled from mersenne.org

From the time when factors could be submitted directly to mersenne.ca it is entirely possible that mersenne.ca might have some additional factors for already-factored exponents that mersenne.org doesn't have, especially since in times past mersenne.org was sometimes reluctant to accept additional factors once an exponent had a factor (this has been changed in the past few months).

If you would like to "harvest" data from mersenne.ca, either one-off or an ongoing basis, let me know and I'm sure I can make something available. Let me know what kind of format and data you would find useful.

[alpertron]

Originally Posted by GP2

* M954257, where Mersenne.ca had four known factors but Mersenne.org had only three. I manually reported the "new" factor at Mersenne.org and got 0.0283 GHz-days credit. Mersenne.ca reports this factor was originally discovered 2015-06-02. It's not clear by whom.

I found that factor. While trying to find factors of known composite Mersenne numbers, I ran P-1 for all exponents 900k to 1M with bounds B1=200K, B2=5M on September 2014 and then with bounds B1=1M, B2=30M on May and June 2015.

[GP2]

Originally Posted by James Heinrich

From the time when factors could be submitted directly to mersenne.ca it is entirely possible that mersenne.ca might have some additional factors for already-factored exponents that mersenne.org doesn't have, especially since in times past mersenne.org was sometimes reluctant to accept additional factors once an exponent had a factor (this has been changed in the past few months).

If you would like to "harvest" data from mersenne.ca, either one-off or an ongoing basis, let me know and I'm sure I can make something available. Let me know what kind of format and data you would find useful.

Send me a private message with an URL, whatever format you have should be fine, I can always run a Python script or something to extract the data. It would be good to complete the consistency checking for all exponents less than 1000M. Although I suspect it would be mostly second-and-higher factors of relatively low exponents that are affected.

[GP2]

Originally Posted by alpertron

I found that factor. While trying to find factors of known composite Mersenne numbers, I ran P-1 for all exponents 900k to 1M with bounds B1=200K, B2=5M on September 2014 and then with bounds B1=1M, B2=30M on May and June 2015.

Hmm, there are lots of other exponents where it did register your factors for those ranges and dates. For instance M950023 for September 2014, M950177 for May 2015, and very many other examples.

How were those results submitted, and did you actually manually submit the results to mersenne.ca? The most alarming scenario would be if you only submitted results to mersenne.org (PrimeNet), which mersenne.ca automatically mirrored as it normally does, and then at some later date the result was somehow dropped from mersenne.org... Do you still have the results.txt or prime.log files that show how the communication with PrimeNet went for that factor?

[James Heinrich]

Originally Posted by GP2

Send me a private message with an URL, whatever format you have should be fine, I can always run a Python script or something to extract the data.

I can send you a full MySQL dump of my factor tables which would include exponent and factor, as well as date-found, bit size, k, etc. Or I can generate a simpler list of exponent,factor if that suffices for your purpose, just let me know what you want.

Originally Posted by GP2

The most alarming scenario ... then at some later date the result was somehow dropped from mersenne.org

I would be extremely surprised if that was the case. I can't think of a scenario that would cause that to happen.

[alpertron]

Originally Posted by GP2

Hmm, there are lots of other exponents where it did register your factors for those ranges and dates. For instance M950023 for September 2014, M950177 for May 2015, and very many other examples.

How were those results submitted, and did you actually manually submit the results to mersenne.ca? The most alarming scenario would be if you only submitted results to mersenne.org (PrimeNet), which mersenne.ca automatically mirrored as it normally does, and then at some later date the result was somehow dropped from mersenne.org... Do you still have the results.txt or prime.log files that show how the communication with PrimeNet went for that factor?

Unfortunately, the machine where I ran the P-1 algorithm with B1=1M, B2=30M had its hard disk broken, so everything was lost.

I tried to send the results to both sites, but since it was a manual method, sometimes I sent the results to only one site. At that moment mersenne.org did not store P-1 results when no factors were found. In March 2016, I sent all my results again (except from that notebook that was broken) to mersenne.org. Madpoo found that in that batch there were some factors there were not submitted to mersenne.org.

[Madpoo]

Originally Posted by alpertron

Unfortunately, the machine where I ran the P-1 algorithm with B1=1M, B2=30M had its hard disk broken, so everything was lost.

I tried to send the results to both sites, but since it was a manual method, sometimes I sent the results to only one site. At that moment mersenne.org did not store P-1 results when no factors were found. In March 2016, I sent all my results again (except from that notebook that was broken) to mersenne.org. Madpoo found that in that batch there were some factors there were not submitted to mersenne.org.

James sent me a factor/exponent list of what he has, so later in the day I'll compare that to what Primenet has and figure out what the differences are. If at all possible, I can try to figure out who found the factor that Primenet is missing and give appropriate credit. That's if I can suss it out.

[GP2]

Originally Posted by Madpoo

James sent me a factor/exponent list of what he has, so later in the day I'll compare that to what Primenet has and figure out what the differences are. If at all possible, I can try to figure out who found the factor that Primenet is missing and give appropriate credit. That's if I can suss it out.

Here's what I have so far (only looked at < 10M for now):

First of all, ignore the date-and-timestamps when comparing the two sets of data, they don't really match up.

New missing factors (I have NOT manually added these). Looks like a whole contiguous batch got dropped at 2.3M:

1076263,404746441851288671009,
1098373,6204639949258669231,
2323939,34907963344957585217,
2324149,10662161439784221823,
2324381,107863054869879369583,
2325181,318915542216651018294231,
2326673,39761566392781203343,
2327099,6427615260255940157417,
2327197,48412896678190509306913,
2327539,8551967281381645001,
2327723,102168863356731488671,
2328707,153988958336242699849,
2328857,6950280843227472409,
2329387,27469302986377283377,
2330099,841016607230679811999,
2330753,14911860910792507153,
2330959,381327229728174339871,
2331047,342543153937917186103,
2331089,52826547962123945702729,
2332313,10541466616863208896809,
2332321,973275814636083173617,
2332373,7967296408733560193,
2332607,6598886445226817393,
2332969,4838699545847273809,
2334139,69752916382675135647606529,
2334301,12054961403420760459737,
7012337,10211694487439188129,

These factors have a trailing space for some reason (shows up when you select "Print simple text report"), there are no others in the factor database < 1000M:

5526161,21131762039662659089 ,
6020621,48921040896521551 ,
10504531,28700796788611131118633 ,

These factors occur twice for some reason (shows up in the Detailed Reports -- Factors Found). Mathematically each factor of a Mersenne number with prime exponent can never be a factor of any other Mersenne number with prime exponent, so maybe the database could create a unique index on the factor field:

2207441,4456310665879544089,
2233019,1219861756779140901119,
2233183,12415622589540644657,

Stuff mentioned in my previous post:

Missing factors for 100003 and 954257, which I already manually added to mersenne.org:

100003,1113838336566049330755578765857,
954257,44522862069024324027649,

Bogus composite factor for 499211 at mersenne.ca, which has now been deleted there:

499211,2253474049908052340696986104536103562940922999791841767712473,

Finally, for some reason mersenne.ca has these (valid) factors for non-prime Mersenne exponents:

100,11,
77777,127,
77777,799993,
77777,595460713,
77777,17137716527,
77777,1398731345881,
77777,60685647225571918873,

[GP2]

New missing factors in the range 10M to 20M, I did NOT manually report these:

10308371,32121976624860440209,
12540691,119576888298349980161,
12667997,5380033915595933471,
14124091,14646676110705643369,
14138779,4420795284601927577,
14160401,13389566679935929601,
14295341,4804614434819289553,
14315201,12085443174681846961,
14554649,5609714823499434887,
15186349,8599602188387049841,

Like the previous batch, these are all second-or-higher factors of previously-factored exponents. For 10308371 and 12667997, these are the seventh known factors (!)

Also, as mentioned in the previous post, 10504531 has a trailing space when you select "Print simple text report" in the Detailed Reports -- Factors Found report:

10504531,28700796788611131118633 ,