After we find a twin, our next exponent will be...

[gribozavr]

I think that two custom scripts or programs (one for starting, one for reporting factors) is the most convenient way if we won't be able to modify newpgen.

Rytis, I see some problems. Sieving a large range requires a lot of RAM. On my Celeron it uses 192Mb, while on Athlon it uses ~65Mb. And, writing a savefile of more than 150Mb is going to be hard-disk intensive and can slow down normal work.

[jmblazek]

Finally, an explaination!!! Thank you axn1.

Rytis, sieving in a week or two! Now that would be amazing. But wait until after Jan 1 as that's when voting for the next n will be closed.

Unfortunately, we only have 16 responses...maybe you can send out email to everyone running the project and ask for them to vote...as a majority of the TPS producers are from PG now.

[MooMoo2]

Originally Posted by gribozavr

It is not difficult to distribute. We just need to write two helper programs:
1. a program which will edit the datafile according to your reservation. A 150Mb+ file can be easily screwed up by manual editing.
2. a program which will find the differences between the original and sieved datafile. It will make a small file with factors, which can be easily sent by email or uploaded somewhere.

You don't need program #2, because NewPGen already has that feature. Go to Options, then click "Log the numbers removed". After that, you'll get a file that says "NewPGen.del", which you can open with wordpad. It'll show all the factors you've found that were not in the original datafile, and it'll look something like this:

p=419328227963 divides k=637391211
p=419329056239 divides k=752861781
p=419343140221 divides k=366268725
p=419352519293 divides k=286648761
p=419374898587 divides k=1008945
p=419390657129 divides k=936394371
p=419410192037 divides k=786378735
p=419419756229 divides k=879549921
p=419434749031 divides k=429345135

---------------------------------

Rytis and Pacionet:

I do not want going to BOINC for sieving. Doing so will take away resources from the main (LLR) task. All that's needed is for a few (4-5) users to sieve, and that's enough. For the current n, we're already quite close to the optimal sieve point, even though sieving was only done by one person with computing power that's only slightly above the TPS average. It might not look that way, since sieving is currently removing slightly more than one k every minute, while an average P4 takes 2 minutes for one LLR test. However, assuming average luck, we'll find a twin at 12-13G. This means that half of those factors are useless, since 13G-25G will never be tested.

The optimal sieving depth for n=333,333 will be slightly bigger than the optimal sieving depth for the current n, but all we need is about 5 users with a decent amount of computing power. The reservations for sieving could either be done manually (posting reservations in a thread), or semi-automatically (when Pacionet's range reservations system is virtually bug-free, it could be modified to handle sieve reservations too).

[axn]

Originally Posted by gribozavr

Rytis, I see some problems. Sieving a large range requires a lot of RAM. On my Celeron it uses 192Mb, while on Athlon it uses ~65Mb. And, writing a savefile of more than 150Mb is going to be hard-disk intensive and can slow down normal work.

Assuming that sieving enters BOINC only after offline sieving has happened to a level > 1P (this is for n=333333 with k range of 1-100G), we might be able to distribute the sieving without a dat file at all!! The big idea is that if you find that a particular k has a factor, you'll check that it is within range and that it doesn't have any "small" factors (small being < 255). [This can be quickly done by trial division and table lookups.] Any k that passes this check will be reported to the server, where it will actually check that the k has not been eliminated previously.

PS:- Naturally, this has to be a custom implementation.

[pacionet]

I agree with moomooo.
Anyway, in the future, the best solution , in my opinion, is:

1) twinprimesearch will be replaced by twinprimesieve.org and will become a site where sievers can upload their pre-sieved files (output of NewPGen)

2) PrimeGrid will handle all the LLR tests on these pre-sieved files

[biwema]

I agree that i does not make much sense to include the sieving into PrimeGrid. (At least for n= 250000 or 333333).
Nevertheless it would be optimal if we have at least 10-20 ahlons dedicated to sieving.

At the moment I am close to 200T and have 38.45 Million candidates left. I still remove more than 1 k per second with a P4 (0.8 second per K). Assuming, we have 200s per LLR, we can go much further.

The save file is 725M (142M zipped). Note that is 3.5 times as large as estimated.
Every hour the file gets 100K smaller. It takes 90 seconds to save the intermediate status.

The task manager reports 300M Memory usage while siving. So I don't recommend sieving with a 512M if you use the computer for other thing while sieving. The suggested minimum memory is 768M or 1G.


If some of you thinks 333333 is too large at the moment, you can start with 250000 first (Sieve a 50G range). I still prefer 333333 and will continue there. Maybe both ranges could be ran at the same time.

I will do my best to bring 333333 to 200T by this weekend. I will make the zipped intermediate file available then.

[MooMoo2]

Originally Posted by pacionet

I agree with moomooo.
Anyway, in the future, the best solution , in my opinion, is:

1) twinprimesearch will be replaced by twinprimesieve.org and will become a site where sievers can upload their pre-sieved files (output of NewPGen)

2) PrimeGrid will handle all the LLR tests on these pre-sieved files

#1 is fine with me.

I have a problem with the word "all" in #2, so I'm only agreeing to it if:

- Primegrid doesn't consider LLR as a sub-project that is in beta-testing. It should first be made stable, so it will have equal status with primegen.

- Primegrid separates LLR tasks from Primegen tasks in the stats. Right now, Skligmund has ~ 134,000 total credit. Is his breakdown closer to:

100,000 credits to primegen, and 33,000 credits to LLR?

or

33,000 credits to LLR, and 100,000 credits to primegen?

It's impossible to know now, so no one can tell how much he contibuted to the project. Partial credit for the twin prime is given to the top TPS producer, so it's important to separate TPS work with primegen work.

- Most (90-95%) of doublechecks are eliminated. All primes should be doublechecked, but checking 5-10% of residues is enough to prevent cheating.

[jmblazek]

Originally Posted by MooooMoo

- Primegrid doesn't consider LLR as a sub-project that is in beta-testing. It should first be made stable, so it will have equal status with primegen.
- Primegrid separates LLR tasks from Primegen tasks in the stats.

Rytis states on the home page: "PrimeGrid is currently running two sub-projects:" Primegen and TPS.

I'm sure that as soon as Rytis feels comfortable that the client is stable, he'll remove the word "testing" from TPS's description. When that happens, hopefully he'll add a feature on the PG preferences page that allows a user to choose which project they want to run...maybe as a percentage of WUs they receive. For example, 10% Primegen / 90% TPS or 100% Primegen / 0% TPS. (FYI...Riesel Sieve still does manual LLR and Sieving...)

However, don't hold your breath for anything soon...as you can see on the home page, the entire PrimeGrid project is still in alpha stage. Is that an oversight or is there really that much development left for PrimeGrid to become stable. Personally, I haven't had a single problem running it straight for almost a month now.

Also, since PerlBOINC seems to be a success so far, I can easily see PG becoming the gateway to BOINC for several other prime searching projects...if that's the direction Rytis wants to go, then PG would have multiple prime searching sub-projects...or possibly other math related projects.

I agree, the credits should be separated out. Riesel Sieve has done it on their charts but have yet to implement it in their stats page. Hopefully the data is easily retrievable for Rytis. I would like to see two additional columns on the stats page which are a breakout of "Total Credit" into "Primegen Credit" and "TPS Credit". From the TPS Credit, it will be easy to determine how many tests have been performed since 0.39 credit is awarded for each test...primes are 0.78 credit. Divide TPS Credit by 0.39 to get # of tests and divide again by 355 (from the LLR Search Status page at PG: 149235 tests/420 M) to get approx. M.

Maybe someone can get Prime95 to view this discussion and offer his suggestions for a way forward with PG, TPS and LLRing/Sieving. He already voted for n between 250,000-300,000.

I chose n=333333 simply for the 100,000 digit attribute. I also thought PG TPS would be crunching faster. Although PG TPS is doing ~6X the previous rate when TPS was manual, it's still about 1/2 way to where it needs to be...1G a week.

p.s. Congrats Rytis for PG finding over 100 primes. In less than a month, PG has already found ~37% of all primes found by TPS. Maybe at a month you'll have 50%!

[Skligmund]

Well, I have around 14,000 credits, in LLR, but that is a guess (not that is matters anymore).

I asked that question in another thread, how will we distinguish total results for LLR in Primegrid? I think this needs to be addressed as a priority, because this is considered a main part of the credit recieved if (when) a Twin prime is found.

[MooMoo2]

Originally Posted by jmblazek

p.s. Congrats Rytis for PG finding over 100 primes. In less than a month, PG has already found ~37% of all primes found by TPS. Maybe at a month you'll have 50%!

The other project milestone is that all ranges under 2G have been reserved:

http://www.mersenneforum.org/showthread.php?t=5818

The stat "progress toward finding a twin" is going to pass 10% soon

[MooMoo2]

Originally Posted by KEP

Actually I hope not, that he decides to do only LLR search, since I hope to see that we will some day have the biggest sequential prime database (which we according to my knowledge already have), going higher than 1,000,000,000,000,000. This will also take many years, but since that is the real no. 1 project, it could be nice to see that happen to.

(bold emphasized by me)

Originally Posted by smh

What is the point of having such a database? Calculating a prime in that range (which is way to small for RSA) is probably much faster than doing a DB lookup

Not only that, but another project has already calculated the primes up to 4*10^22. In fact, they have even counted the number of primes between 2 and 4*10^22 (there are 783,964,159,847,056,303,858 primes under 4*10^22, if anyone's curious).

See:

http://numbers.computation.free.fr/C...ixproject.html

for more info on that project. On the other hand, Primegrid hasn't even completed a sequential prime database that contains numbers higher than 10^11 (according to the page
http://www.primegrid.com/orig/torrent.php)

edit: I read the project description further and realized that you don't have to calculate the actual primes in order to know the number of primes below a certain number. Oh well, it doesn't matter too much, since another project proves my point that Primegrid won't have the largest sequential prime database for a long time. The project that has the largest sequential prime database was one that estimated Brun's constant, and it was completed a few years ago:

Originally Posted by Pi(x) project website

This sieve is now classical and was extensively used for example as a distributed projet on the net to compute all prime numbers less than 10^16

Rytis, I'm not going to be overly persuasive in suggesting you to reconsider primegen. It's you project, so you're free to do whatever you want with the primegen application. However, you should be aware that the work primegen is doing has already been done before. If you decide to continue making a list of all the small primes, my suggestion would be to use this program on this page instead:

http://www.rsok.com/~jrm/printprimes.html

The key phrase is:

old 300 MHz AMD K6 took 1 hour and 33 minutes just to print the 367783654 primes less than 8,000,000,000.

According to your website http://www.primegrid.com/orig/torrent.php, you are now at 35,000,000,000. Using this program means that you can double that number to 70,000,000,000 with less than a few hours of work.