Prime gap records

[CraigLo]

Sorry, I think I caused the server to hang (Queue 76: Testing 12968, Endpoints tested, sieving interval) with this submission.


12968 29.2784 162798848363269810374491*421#/210 -3846


Maybe the multiplier is too large. I didn't have my code set up correctly which resulted in the large multiplier but I happened to find this record in the first minute. No worries if you don't want to handle this case.

[SethTro]

Originally Posted by CraigLo

Sorry, I think I caused the server to hang (Queue 76: Testing 12968, Endpoints tested, sieving interval) with this submission.

12968 29.2784 162798848363269810374491*421#/210 -3846

Maybe the multiplier is too large. I didn't have my code set up correctly which resulted in the large multiplier but I happened to find this record in the first minute. No worries if you don't want to handle this case.

I had an artificial limit on the multiplier to be small than 2^62, not sure why. I removed it now (maybe I should have increased to 128 bits but muh).

https://github.com/sethtroisi/prime-...ad15d57ebffbed

I kicked the server and re submitted your batch so you should be good.

[SethTro]

Originally Posted by CraigLo

I found 3 pretty large merits that are just outside the top 100.

10470 33.7912 17497330955201*307#/30 -6066
10470 33.7896 17755704968891*307#/30 -7496
10470 33.7881 18011066308391*307#/30 -4196

There is a larger merit for 10470. Is there a record for largest merit that is not a record?

I was curious what other large record updates have happened

# Ugly hack to get which updates aren't really updates

#$ time git log --pretty="tformat:%h" --shortstat | tqdm > sizes
# Look for format and name changes

#$ grep -B2 -E '[0-9]{4} insertions' sizes | grep '^[a-f0-9]' | xargs git show -s --pretty=short

# Search diffs for updates with very large merits
$ time git show -p $(git rev-list --all | grep -Pv '^(ba884d39|b63c9327|09839e0|c307be0b|cf5e6e30|50e69185)') | tqdm > test
740258it [01:12, 10259.52it/s] 

$ cat test | grep -A8 -P '^\-.*,3[3-9]\.[0-9]*'
-INSERT INTO gaps VALUES(12496,0,'C','?','P','Loizides',2021,33.1768,164,'657650577373*379#/30-3750');
+INSERT INTO gaps VALUES(12496,0,'C','?','P','Loizides',2021,33.1982,164,'1368497503950671*373#/210-6382');
--
-INSERT INTO gaps VALUES(12552,0,'C','?','P','Loizides',2021,33.3265,164,'650096530123*379#/30-8654');
+INSERT INTO gaps VALUES(12552,0,'C','?','P','Loizides',2021,33.3368,164,'1535469329111651*373#/210-6268');
--
-INSERT INTO gaps VALUES(9738,0,'C','?','C','Jacobsen',2016,33.4953,127,'7850579863*283#/30 - 5534');
+INSERT INTO gaps VALUES(9738,0,'C','?','P','Gapcoin',2021,33.6448,126,'501659213896641070425551073826394741758182328582700514152199821460191935224220593522615735518498430438019998041878981782695809');
--
-INSERT INTO gaps VALUES(1490,0,'C','?','C','Be.Nyman',2011,33.609999999999999431,20,'17849040361018364489');
+INSERT INTO gaps VALUES(1490,0,'C','F','C','Jacobsen',2017,34.280000000000001136,19,'7562321651334729221');

None of which are as large as the three you proposed.

[SethTro]

Hey everybody.

I wanted to circle back on large gaps and the expectations from https://primegaps.cloudygo.com/

As a reminder all of this code is open-source on github.

I'm never going to be able to verify all records especially large records. The server needs to double check all work that result in records. For small records this is a tiny fraciton (one millionth) of the work done and easy to verify. For large records this can be 10% or more of the work; if you find the surrounding primes for say 1009 * 200003#/2310 it's likely to generate a low merit result but the server will not have seen the large gap before and will be forced to check hundreds of thousands of large primes.

For records > 22 merit the server verifies the whole record, in the case that this takes a long time it's likely the server will die and the record won't be recorded (and it's on you to notice and resubmit / email me)

If the merit is < 22 the server calculates the expects time to verify a record, if that is greater than three minutes, the server only partially checks the record but marks the record as C?? in allgaps.sql. C?? = Common, not first occurrence, not completely verified. This works great for records ~3000 digits up to ~28000 digits. A progressively smaller amount of the record gets verified but that's noted by the C?? status.

Currently there are 14,250 (~13%) C?? records, the smallest has gap=95864 (and merit 15) the largest has gap=1085478

For mega records (>28,000 digits) the server can probably test the endpoints but I'm not going to be able to verify the interior. Twice formunites have verified a record then I manually add it to the allgaps.sql file. I'd prefer to not do that going forward.

---

So there are a couple of open questions:
1. Are we happy with 14% of records being "C??" AKA not verified ?
2. Should I blindly accept larger records (>28,000 digits) after checking the endpoints?
3. Would weekly updates for https://primegap-list-project.github.io/ work for people?
4. Anything else the server could do that people would appreciate?

[MJansen]

Hi Seth,

the principles seem sound. My 2 cents regarding your questions:
1. Are we happy with 14% of records being "C??" AKA not verified ?
Yes

2. Should I blindly accept larger records (>28,000 digits) after checking the endpoints?
Yes, with the addition that if a false result is found later, the result will be scraped and possible other gaps will be reviewed, which could ultimately result in black-listing the perpetrator (or something along these lines)

3. Would weekly updates for https://primegap-list-project.github.io/ work for people?
Yes, I guess especially the tables are updated rather irregular, and for the casual visitor this might seem illogical. The source data is up to date however, so no real need to update these pages in that respect. But it is nice to see the record tables updated more frequently.

4. Anything else the server could do that people would appreciate?
Not at this moment if I speak for myself.

Kind regards and thanks again for the good work!
Michiel

[robert44444uk]

Originally Posted by SethTro

So there are a couple of open questions:
1. Are we happy with 14% of records being "C??" AKA not verified ?
2. Should I blindly accept larger records (>28,000 digits) after checking the endpoints?
3. Would weekly updates for https://primegap-list-project.github.io/ work for people?
4. Anything else the server could do that people would appreciate?

My tuppence worth,

1. Yes, other than a few big results, these records can also be broken easily I would say that it would be good if those that are merit 20 or above are checked
2. Yes, if from a member of this group, who are using programs that have been proved to work all of the time
3. Agee with Michiel's comments
4. The two that I think could be useful is a downloadable .txt file of the records, which will be 11-12 MB. This was always available on the Dr Nicely's site and (ii) something that shows the date of the last time the visible github page was updated.

[MJansen]

Originally Posted by robert44444uk

My tuppence worth,

1. Yes, other than a few big results, these records can also be broken easily I would say that it would be good if those that are merit 20 or above are checked
2. Yes, if from a member of this group, who are using programs that have been proved to work all of the time
3. Agee with Michiel's comments
4. The two that I think could be useful is a downloadable .txt file of the records, which will be 11-12 MB. This was always available on the Dr Nicely's site and (ii) something that shows the date of the last time the visible github page was updated.

Based on a sample of two biased contributions:
1. Agreed by the contributers
2. If a known contributer sends in a big gap it will be accepted. A new contributer will be asked to send in his possible big gap to this list. End points can be verified by one of the members (that does not take that long). A sample of the intermediate candidates can be reproduced by me, takes around 1 hour to have the interval sieved till 10^10. Selecting some candidates to be tested, and after completing will give the C?? certificate. After that the new contributer is accepted and can submit prime gaps. Occasional tests can be conducted and faulty submissions will be strikken.
3. Agreed by the contributers
4. @Robert: There is already a TRN-like list (https://primegaps.cloudygo.com/merits.txt) made a while ago on my request, and an all-gap list (https://raw.githubusercontent.com/pr...er/allgaps.sql) maybe the reference to these can be included somewhere?
I second the wish for a timestamp of the last update on the pages (if possible).

Adequate response for now?
Kind regards
Michiel

[robert44444uk]

Originally Posted by MJansen

Based on a sample of two biased contributions:

4. @Robert: There is already a TRN-like list (https://primegaps.cloudygo.com/merits.txt) made a while ago on my request, and an all-gap list (https://raw.githubusercontent.com/pr...er/allgaps.sql) maybe the reference to these can be included somewhere?
I second the wish for a timestamp of the last update on the pages (if possible).

Michiel

Not everyone is comfortable with .sql files, .txt is well understood, and a .zip or .7z or well known compression compression can keep the file size down

It would be good to have all of the results, tables etc on either the prime-gaps-github or the the cloudy.go site i.e. one stop shop. At the moment there is the prime-gap-list github where all the results are, tables, graphs etc and cloudy go for uploading. Maybe I am not up to date on where things are!!

[MJansen]

Originally Posted by robert44444uk

Not everyone is comfortable with .sql files, .txt is well understood, and a .zip or .7z or well known compression compression can keep the file size down

It would be good to have all of the results, tables etc on either the prime-gaps-github or the the cloudy.go site i.e. one stop shop. At the moment there is the prime-gap-list github where all the results are, tables, graphs etc and cloudy go for uploading. Maybe I am not up to date on where things are!!

Hi Robert,

Both links appear in my webbrowser (Ms Edge) and then I copy the content of the webpage to Excel, so a work around for sure. I agree a more neat and standardized solution would be preferable. In my own selfish way, I can live with the current one.
About not knowing where things are: same here, I just know the links because I have them bookmarked ;-)

@Seth, you have enough input?

Kind regards
Michiel

[SethTro]

Originally Posted by MJansen

Based on a sample of two biased contributions:
4. @Robert: There is already a TRN-like list (https://primegaps.cloudygo.com/merits.txt) made a while ago on my request, and an all-gap list (https://raw.githubusercontent.com/pr...er/allgaps.sql) maybe the reference to these can be included somewhere?
I second the wish for a timestamp of the last update on the pages (if possible).

Adequate response for now?
Kind regards
Michiel

I agree that a timestamp would be nice. I'm not sure how best to add it as the github site originated from Graham Higgins but I'll make an attempt.
It seems I can do this https://ryanfb.github.io/etc/2020/04...yll_posts.html but kinda ugly
I will try with a git precommit hook (https://stackoverflow.com/a/33721446/459714) and see if that works


There's also a txt version of the allgaps.sql at https://raw.githubusercontent.com/pr...ta/allgaps.csv note that it's only updated weekly vs the sql file that is updated on each submission.

[MJansen]

Originally Posted by SethTro

I agree that a timestamp would be nice. I'm not sure how best to add it as the github site originated from Graham Higgins but I'll make an attempt.
It seems I can do this https://ryanfb.github.io/etc/2020/04...yll_posts.html but kinda ugly
I will try with a git precommit hook (https://stackoverflow.com/a/33721446/459714) and see if that works


There's also a txt version of the allgaps.sql at https://raw.githubusercontent.com/pr...ta/allgaps.csv note that it's only updated weekly vs the sql file that is updated on each submission.

Hi Seth,

have you been able to add the option to enter large gaps? I have a nice one to add (was hoping for more of course):
5198448 14.335331 363257*363269#/510510-2177806

Kind regards
Michiel