mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > No Prime Left Behind

Closed Thread
 
Thread Tools
Old 2008-02-28, 17:07   #12
em99010pepe
 
em99010pepe's Avatar
 
Sep 2004

2·5·283 Posts
Default

After a few minutes of wise thinking I decided to leave this conversation. I really don't care how many servers will be available when I can manually reserve ranges. I'm hosting 2 NPLB servers, after they dry I am out. I'm reaching a point where I am tired of human stupidity so I prefer to walk away and silently crunch the manual ranges.

Carlos

Last fiddled with by em99010pepe on 2008-02-28 at 17:08
em99010pepe is offline  
Old 2008-02-28, 17:10   #13
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

5×2,017 Posts
Default

Quote:
Originally Posted by em99010pepe View Post
With all these option servers with big range of work when are you guys thinking to fill all gaps, in 2020? The work will be dispersed...

So far we have three servers running lower ranges and one with Top 5000 candidates. Let's dry the first three ones, shall we? Stop talking of new servers, we only have 100 cores at our disposal and not 1000!!! One step at the time, not two...

Carlos
It was my intent with this project to allow people a choice and to fill in the lower range as we searched for top-5000 primes. I'd still like to maintain that. Initially, the higher priority was the higher n-range but as we got out of the danger zone of primes > 333.3K not being in top 5000, it has shifted. In other words, at first 75/25% on higher range...now ~60/40% on lower range.

k=300-400 will be the same way. Higher priority on higher n-range at first for same reason followed by higher priority on the lower n-range to clean it up.

I wouldn't want to dictate to people that they have to search the lower range until we get the 'mess' cleaned up.

The main thing that I'm the happiest about is that people WANT to clean up the lower-range 'mess'. It was a concern of mine when the project started that it would be difficult to get people to search non-top-5000 ranges. THANKS TEAM!!


Gary
gd_barnes is offline  
Old 2008-02-28, 17:32   #14
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

5×2,017 Posts
Default

I renamed this thread to 'Follow up on LLRnet servers needed' for future historical reference.
gd_barnes is offline  
Old 2008-02-28, 17:51   #15
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3×2,083 Posts
Default

Quote:
Originally Posted by em99010pepe View Post
After a few minutes of wise thinking I decided to leave this conversation. I really don't care how many servers will be available when I can manually reserve ranges. I'm hosting 2 NPLB servers, after they dry I am out. I'm reaching a point where I am tired of human stupidity so I prefer to walk away and silently crunch the manual ranges.

Carlos
Okay, I can understand that. You do have a point, we don't exactly NEED LLRnet servers for every single subproject and drive. Even the big projects such as PSP and SR5 do fine with only one or two servers.

Okay, here's a new idea I'm putting up for discussion:

Maybe here's what we should do: have one LLRnet server for a higher range, one LLRnet server for a lower range, and maybe one server for the n>600K testing later on. (The doublecheck server should probably be counted separately, because it's dealing with a wholly different type of work.) We don't necessarily need a server for each of those ranges on both 300<k<400 and 400<k<1001; instead, maybe we should have those three respective servers doing whichever k-range needs the most work for their designated n-range.

As long as we host our servers on machines such as IronBits' that can handle the stress of a rally without breaking a sweat, then we shouldn't need "backup" servers on any of the ranges.

That would keep logistical issues to a minimum, and avoid spreading our resources too thin, while still giving people plenty of choice as to n-ranges. Remember, as Mini-Geek said, most people won't care whether they're doing 300<k<400 or 400<k<1001, though they will care which n-range they're doing.

What does everyone think?

Anon
mdettweiler is offline  
Old 2008-02-28, 18:43   #16
Flatlander
I quite division it
 
Flatlander's Avatar
 
"Chris"
Feb 2005
England

31×67 Posts
Default Double check server

(There's so much server discussion on this forum so forgive me if this has been said.)

At the beginning of the double check, cores could be returning results every few seconds. Would a server cope and how much time would the client waste with all the comms?
Would reserving work units be better at the beginning?
Flatlander is offline  
Old 2008-02-28, 19:23   #17
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

5·2,017 Posts
Default

Quote:
Originally Posted by Mini-Geek View Post
I think that 300<k<400 and 400<k<1001 are similar enough that they should be put together on the server. Come to think of it, I agree that the doublecheck is different enough that I think it warrants its own server.
Edit: So I now suggest three distinct: Top 5000, first-pass non-Top 5000, and Doublecheck non-Top 5000, with probably 2 servers on the first two distinct groups. You can work out which servers run what.
Part of my reasoning for having these three is the sort of work preference that people would have, while still being easy. After all, would someone that wanted the ease of LLRnet likely care if they got 300<k<400 work or 400<k<1001 work? I think not. Would they care if they were doing top 5000, and if they were running doublechecks compared to first-pass? I think so.
If part of the 300<k<400 is to be put into a manual reserving team drive, I think it should be a separate drive.

Quote:
Originally Posted by Anonymous View Post
Okay, I can understand that. You do have a point, we don't exactly NEED LLRnet servers for every single subproject and drive. Even the big projects such as PSP and SR5 do fine with only one or two servers.

Okay, here's a new idea I'm putting up for discussion:

Maybe here's what we should do: have one LLRnet server for a higher range, one LLRnet server for a lower range, and maybe one server for the n>600K testing later on. (The doublecheck server should probably be counted separately, because it's dealing with a wholly different type of work.) We don't necessarily need a server for each of those ranges on both 300<k<400 and 400<k<1001; instead, maybe we should have those three respective servers doing whichever k-range needs the most work for their designated n-range.

As long as we host our servers on machines such as IronBits' that can handle the stress of a rally without breaking a sweat, then we shouldn't need "backup" servers on any of the ranges.

That would keep logistical issues to a minimum, and avoid spreading our resources too thin, while still giving people plenty of choice as to n-ranges. Remember, as Mini-Geek said, most people won't care whether they're doing 300<k<400 or 400<k<1001, though they will care which n-range they're doing.

What does everyone think?

Anon
I can go along with these ideas. This is my fault here. It was my ignorance of what constitues a 'large' prime search project when I made the initial suggestion of servers. Heck, my thinking is we are a large prime search project with 100+ cores searching and 50+ top-5000 primes in the first month. But clearly 100 cores does not constitute large in the prime-searching community.

We would want a server specifically on k=300-400 for n>333.3K for a rally the weekend of the 8th. But when n=300-400 'catches up' with n=400-1001, I can merge the big sieve file together and we can all work in harmony on the top-5000 range by itself with a single server for it (with a backup available if necessary). It may catch up with just the rally because it's < 1/6th as many k's. So perhaps temporarily, we could have 2 servers for n>333.3K; one for each k-range, with the idea of merging the range later. Or...as Anon said, just move the high-range server to k=300-400 until it gets caught up and not have one for k=400-1001.

For k=400-1001 n<333.3K, we could just leave one server on that range and when it's done, move the server over to k=300-400 n<333.3K until it is done.

Quote:
Originally Posted by Flatlander View Post
(There's so much server discussion on this forum so forgive me if this has been said.)

At the beginning of the double check, cores could be returning results every few seconds. Would a server cope and how much time would the client waste with all the comms?
Would reserving work units be better at the beginning?
It hadn't been brought up but I alluded to it when I offered my future server for double-checking (in another thread I think). You may be correct, we may want to do manual LLR reservations for a while. Alternatively, we could just simply ask people to limit the # of cores that they put on it.


Gary
gd_barnes is offline  
Old 2008-02-28, 20:14   #18
Mini-Geek
Account Deleted
 
Mini-Geek's Avatar
 
"Tim Sorbera"
Aug 2006
San Antonio, TX USA

102538 Posts
Default

Quote:
Originally Posted by Flatlander View Post
and how much time would the client waste with all the comms?
Would reserving work units be better at the beginning?
As long as the client had it set to cache at least, say, five (preferably higher), there wouldn't be any waste, assuming the server could take it. I doubt the server could take it, so I agree with Gary that we should do manual LLRing until it gets to a reasonable amount (I don't know FFT cutoffs for things this small, so I don't know when we could get it to an amount a server could handle), then maybe LLRnet/rally away the rest...or just keep doing it manual, if we prefer that when the time comes.
Quote:
Originally Posted by gd_barnes View Post
We would want a server specifically on k=300-400 for n>333.3K for a rally the weekend of the 8th. But when n=300-400 'catches up' with n=400-1001, I can merge the big sieve file together and we can all work in harmony on the top-5000 range by itself with a single server for it (with a backup available if necessary).
Would it be very difficult (or otherwise problematic) to sort the k=300-400 range into the knpairs.txt, then host a rally on that server? That way, once it's caught up, it'll start integrating the two ranges automatically and immediately.
I don't know how hard OOo or Excel would choke on a file as large as what we're dealing with, but you could always dump it in (make it .csv then make a space be the divider, or find/replace the space by a comma...this also works for looking at timings in lresults.txt), then sort it that way...Or you could just have it sort where the two ranges overlap, instead of dealing with the entire thing at once.
Mini-Geek is offline  
Old 2008-02-28, 20:38   #19
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

5×2,017 Posts
Default

Quote:
Originally Posted by Mini-Geek View Post
As long as the client had it set to cache at least, say, five (preferably higher), there wouldn't be any waste, assuming the server could take it. I doubt the server could take it, so I agree with Gary that we should do manual LLRing until it gets to a reasonable amount (I don't know FFT cutoffs for things this small, so I don't know when we could get it to an amount a server could handle), then maybe LLRnet/rally away the rest...or just keep doing it manual, if we prefer that when the time comes.

Would it be very difficult (or otherwise problematic) to sort the k=300-400 range into the knpairs.txt, then host a rally on that server? That way, once it's caught up, it'll start integrating the two ranges automatically and immediately.
I don't know how hard OOo or Excel would choke on a file as large as what we're dealing with, but you could always dump it in (make it .csv then make a space be the divider, or find/replace the space by a comma...this also works for looking at timings in lresults.txt), then sort it that way...Or you could just have it sort where the two ranges overlap, instead of dealing with the entire thing at once.
Actually, that would be no problem at all. I'd just have to use srfile with the -G parameter. It would automatically sort it by n. There would be too many rows to sort it in Excel.

Interesting idea. I like it. The only issue is that before we did that, I think we'd want to run the current LLRnet servers dry so that we aren't sitting there with a bunch of gaps for k=400-1001. I don't think we could finish that up by the 8th because most people are on n<320K right now.

So to incorporate and summarize your idea, here's what I think we'd need to do:
1. Have a temporary 2nd high-range server that exists only for the rally for k=300-400 n>333.3K AND until no more gaps exist in current LLRnet reservations for k=400-1001 n>333.3K.
2. After #1 is done, regardless of what n-range the two k-ranges are at, merge the two together and continue. Since they will be sorted by n, the k-range that has been searched the least will be searched until it is caught up.


Gary
gd_barnes is offline  
Old 2008-02-28, 21:50   #20
tnerual
 
tnerual's Avatar
 
Oct 2006

7·37 Posts
Default

here are my 2 cents ... (i forgot to read everything above)

in the long term, the project only need 2 server
  • first check
  • double check

to go to this place, we need to level up all files to join the same level !

first thing to do:
sieve all sieve files to the same level so that we can merge all files to make only one big file.

we don't really need more than 2 server in every case.
we don't need to split the load if server are stable (IB one seems to handle the load)

at the moment, the two stable servers needed are:

- n=>333333 to find top 5000 primes (in range 1<k<1001)
- n=<333333 to complete all the gaps in range 1<k<1001

once all the gaps bellow 333333 are filled, we can switch this server to release double check tests ... (not really important at the moment)

this will limit the numbers of server to a maximum of 2 and the choice will be clear for everybody:
2 choices: top5000 primes or "clear Gary's mess" (this last one changing over time to "delation: mr X didn't check this file perfectly")

i think that's what carlos was trying to explain.

the division in k ranges (0-300 300-400 400+) is not usefull for llring (just complicating the reservation pages.) it's usefull for sieving (sr2sieve can't manage 500+ k numbers) but only for sieving !

we have to equalize everything to have a clear project !


as for the double check server: i will start one here limited to my home farm, it will clear all the low values (fast results hitting a server). once Gary's mess is cleared (every k/n pairs below n= 333333 tested once for every k< 1001) i will release the knpairs.txt file to the official llrnet double check server owner to let everybody crunch it.

but first: CLEAR GARY'S MESS

Last fiddled with by tnerual on 2008-02-28 at 21:57
tnerual is offline  
Old 2008-02-29, 00:01   #21
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

22×691 Posts
Default Ex*el and limits

about Ex*el and the row-limits:
you have to think 3-dimensional.
i know E* can only load 65535 rows in one spreadsheet but what about more than one?

i make my VBAscripts to summarize the k/n-pairs for users/teams, ranges and daily stats in such a manner that all spreadsheets named 'ResultsXXX' are summurized. before that i convert the LLRnet pairs in semicolon-seperated text files with the data i need with gawk (easy and quick). import this in E* and call the macro: that's all. and without a DB it's the fastest way i can do. and the most independed one. the most work i got to do is to eliminate/correct 'false' usernames and make the changes in the html-pages. the last could be done by creating a spreadsheet/textfile to write the data i need in html-code. that's my next step to do.
kar_bon is offline  
Closed Thread

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
LLRnet and PRPnet servers for automated LLR mdettweiler Twin Prime Search 229 2020-03-29 06:53
LLRnet servers for NPLB kar_bon No Prime Left Behind 1343 2014-08-20 09:38
LLRnet servers for CRUS gd_barnes Conjectures 'R Us 39 2008-07-15 10:26
New LLRnet servers discussion IronBits Conjectures 'R Us 11 2008-03-20 03:43
LLRnet servers needed gd_barnes No Prime Left Behind 63 2008-03-01 03:36

All times are UTC. The time now is 01:52.

Tue Mar 31 01:52:09 UTC 2020 up 5 days, 23:25, 0 users, load averages: 1.77, 1.64, 1.55

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.