Future project direction and server needs synopsis

[gd_barnes]

After hearing everything, I think we now need a clear-cut decision on project direction and the need for servers. Here is what I think our project and server needs are now and in the future:

In coming up with these needs, I see things occurring in this sequence:
(1) Completion of sieving of k=300-400 for n=260K-600K; will be done late Friday.
(2) Rally occurs for k=300-400 higher range and we begin to dry up the k=400-1001 higher range server.
(3) The k=400-1001 higher range server runs dry and we merge k=300-400 top range with k=400-1001 top range.
(4) Completion of k=400-1001 lower range.
(5) Double checking is ready for n=100K-260K after initial manual LLRing at lower ranges.
(6) Completion of k=300-400 lower range.
(7) Completion of sieving of k=300-400 for n=600K-1M.
(8) Completion of double checking n=100K-260K.

Before (1):
Server A on top range k=400-1001
Server B on on lower range k=400-1001
(2 total)

After (1) and before (3); rally occurs in the middle of this:
Server C on top range k=300-400
Server A on top range k=400-1001
Server B on lower range k=400-1001
(3 total)

After (3) and before (4):
Server C on top range k=300-1001 (merged k=300-400 & k=400-1001)
Server B on lower range k=400-1001
(Could also run a 3rd server on lower range k=300-400 but not necessary at this point)
(2 total)

After (4) and before (5):
Server C on top range k=300-1001 (now merge of k=300-400 & k=400-1001)
Server B on lower range k=300-400 (changed from k=400-1001)
(2 total)

After (5) and before (6):
Server C on top range k=300-1001
Server B on lower range k=300-400
Server A on double checking
(3 total)

After (6) and before (7):
Server C on top range k=300-1001
Server A on double checking
(2 total)

After (7) and before (8):
Server C on top range (n=333K-600K) k=300-1001
Server B on n>600K k=300-400 (only ~10 k's already searched as high)
Server A on double checking
(3 total)

After (8):
Server C on top range (n=333K-600K) k=300-1001
Server B on n>600K k=300-400

After the above is done, we can think of sieving k=400-1001 for n>600K and possibly a little later, double checking n=~260K-400K.


Carlos has one server on top range and one on lower range for k=400-1001. If he no longer wishes to host those, then we can decide who will take them; most likely Iron Bits. Also Adam has a server that he is going to run dry now that would be available in the future.

Can everyone live with at most 4 servers running at the same time with us eventually getting down to 2 or 3? I say 4 even with a max above of 3 because there could easily be some overlap of the completion of different things. Anon/Carlos/Karsten, do you feel it would be reasonably maintainable as shown above?

I'd like to head towards this because I started the project with the thinking that it would be nice if it appealed to the most people possible by allowing searching of different n-ranges and both manual and automated LLR processing.

In having a hybrid project of this nature, there is a mixture of manual and LLRnet reservations available at 2-3 different n-ranges mixed in with double-checking. I think it makes for more fun and more possibilities for rallies!

In other words, I want to avoid the tendency to gear towards a 'generic' project where there's only one range to search with one server to do so for first-pass processing. I'm looking for a 'hybrid' of a project in between completely manual RPS and completely automated RieselSieve/SOB.



Gary

[mdettweiler]

Sounds good to me. One thing, though: as was discussed in the "follow up on llrnet servers needed" thread, the doublechecking for n=100K-260K shouldn't have any LLRnet server until it gets to at least about n=200K or so. Until then, all the overhead will probably be too much for any server to handle--and remember, we want to be careful with which computers we have doing doublechecking for n=100K-260K, because all the results have to be completely accurate--i.e. no machines with stability that's at all questionable. (When we get above n=260K for 300<k<1001, of course, since we'll have first-pass residuals to compare with, any machines can be used on the doublechecking; however, for anything below n=260K for 300<k<1001, and all of k<300, we have to be completely sure of our doublecheck results.)

Anon

[TimSorbet]

I like the plan. Few servers, kept nice and simple.

Originally Posted by Anonymous

Sounds good to me. One thing, though: as was discussed in the "follow up on llrnet servers needed" thread, the doublechecking for n=100K-260K shouldn't have any LLRnet server until it gets to at least about n=200K or so.

Yeah, he put that in.

(5) Double checking is ready for n=100K-260K after initial manual LLRing at lower ranges.

Originally Posted by Anonymous

Until then, all the overhead will probably be too much for any server to handle--and remember, we want to be careful with which computers we have doing doublechecking for n=100K-260K, because all the results have to be completely accurate--i.e. no machines with stability that's at all questionable. (When we get above n=260K for 300<k<1001, of course, since we'll have first-pass residuals to compare with, any machines can be used on the doublechecking; however, for anything below n=260K for 300<k<1001, and all of k<300, we have to be completely sure of our doublecheck results.)

Anon

Perhaps it's best not to have that low range DCing on LLRnet then, because we want people to be involved enough that they shouldn't put questionable machines on it. I think we should only have DCing on LLRnet for ones where residues are ready.

[gd_barnes]

Originally Posted by Anonymous

Sounds good to me. One thing, though: as was discussed in the "follow up on llrnet servers needed" thread, the doublechecking for n=100K-260K shouldn't have any LLRnet server until it gets to at least about n=200K or so. Until then, all the overhead will probably be too much for any server to handle--and remember, we want to be careful with which computers we have doing doublechecking for n=100K-260K, because all the results have to be completely accurate--i.e. no machines with stability that's at all questionable. (When we get above n=260K for 300<k<1001, of course, since we'll have first-pass residuals to compare with, any machines can be used on the doublechecking; however, for anything below n=260K for 300<k<1001, and all of k<300, we have to be completely sure of our doublecheck results.)

Anon

See (5) in the sequence of events.

[mdettweiler]

Originally Posted by Mini-Geek

I like the plan. Few servers, kept nice and simple.
Yeah, he put that in.
Perhaps it's best not to have that low range DCing on LLRnet then, because we want people to be involved enough that they shouldn't put questionable machines on it. I think we should only have DCing on LLRnet for ones where residues are ready.

Yes, I agree. I was at first thinking that maybe we could put the doublecheck LLRnet server on a "secret" port and you'd have to PM or email me in order to get it, but yeah, you're right, it probably would be better just to do the whole thing manually.

Come to think of it, the manual LLR application is faster anyway--LLRnet is based on LLR 3.5.0, and there were some major (i.e. 3-5%) speed increases in LLR 3.6.0. So, I guess it probably is best we do manual LLR only for the n=100K-260K doublechecking, and then do LLRnet for it once we get past 260K.

[mdettweiler]

Originally Posted by gd_barnes

See (5) in the sequence of events.

Ah, I see, I missed that the first time through.

[gd_barnes]

Originally Posted by Mini-Geek

I like the plan. Few servers, kept nice and simple.
Yeah, he put that in.
Perhaps it's best not to have that low range DCing on LLRnet then, because we want people to be involved enough that they shouldn't put questionable machines on it. I think we should only have DCing on LLRnet for ones where residues are ready.

This would be a possiblity. Let's revisit it when manual LLR is @ n>150K on double-checking. If it's just 10-20 cores hitting a server, even a low range shouldn't make the server barf. 10-20 cores at n=150K would be like 40-80 cores at n=300K. 2x the n-range = 4x the processing time. We may want to wait until n=200K if at all depending on how fast we're getting it done.