Replacing Entropia's primenet server

[Prime95]

Aga seems to worry about bogus results being returned from a modified client to pad one's stats.

I and I think Scott are worried about runaway clients eating through all the available exponents in a hurry.

Both concerns are valid, we will have to carefully examine each message that can be sent to the server and look for ways to minimize or eliminate possible damage from modified clients.

We should be able to identify fake LL results without requiring triple-or-more-checking. One new feature I want to add is to make sure the double-check is sent to a different userid and different team. We should also make sure that at least one LL result comes from a trusted prime95 client - this should be easy as 99% of results come from prime95/mprime executables built by me.

[QuintLeo]

I don't know if the current server does this or not - but a DC assignment should *never* be assigned to the same user that did the first time check - and I think it should never be assigned to a teammate, if teams are allowed to swap exponents around.

[Old man PrimeNet]

Originally Posted by aga

It sounds easiest to deliver data from core server(s) to slave stats server(s) using realtime MySQL replication. ... MySQL replication allows to filter out data unneeded by stats servers - sounds like it does not cause security problems in this particular case. (Hereby I assume that there will be at least one core server that runs with a private MySQL engine;

I read that mySQL replication can stream transactions to several peer servers concurrently. Can one filter for stats reporting, and the others as failovers not filter?

Originally Posted by aga

If don't filter replication log, then stats servers will also act as backups (even failovers?) for core servers, thus eliminating need of explicit backing up.;

... Unless a corruption propagated and brought all the servers down. Then an occasional checkpoint backup sounds good again. Disk is relatively cheap, and until we are super-confident with the system all backup options are potential safety nets.

Passwords in stats servers: No big deal? To me, the biggest risk having more servers in the picture is we might get sloppy with our maintenance procedures or server install/change configurations. If done right, however, I don't see much concern about stats servers having login passwords mirrored and used for detailed account viewing. You are right they would need to be protected within a trusted server, though. The v2-v4 systems have been like this for 5+ years, maybe not the best precedent but it has been ok so far.

Security mechanisms cont. - I like the 32-bit key-per-assigned-exponent idea. Maybe we can do that for the clients, too ... I think someone else said that already - and revoke the access key if we think they are misbehaving?

Originally Posted by Paulie

I really like the idea of adding a "security code" to an assignment. Something like an MD5/SHA of a date/time+random seed+M(xxx) should work well.

A random 32-bit int as a secret key is just as sufficient. Why do more?

Throttling open-source client transaction rates - George is right that I'm mostly concerned with runaway/maliciously-coded clients. How do we propose handling someone exceeding a throttle rate limit? If they are getting all the way to the main database to decide someone is ok or not, we are taking the transaction hit already.

Here's an idea to whack at - What if throttling was managed by the web layer above the database? (Remember that the web layer can reside on the same server as the database, or on a different one.) The current v4 web layer authenticates the message hash signature of trusted clients. A v5 web layer could maintain a local, self-populated cache/table that tracked hit rates by IP & Unique Machine ID ("MUID"?), and respond by passing the transaction hit normally, ignoring it, revoking its client key (is that evil?) or replying with an error message - all without hitting the main database. The database would be self-maintaining in the sense that it records and gates access, but otherwise does not need loading or purging of data from the main v5 database. Actually I believe throttling can be router or firewall configured, too - any experts?

JSP/Servlets technology - There are no requirements forcing a choice of Java over any other option. I've led a number of Java projects and believe me it's no cure-all as advertised.
1. First, the right design will make the choice of stateless http-to-database interface, JSP, ASP, CGI, ISAPI, etc., irrelevant anyway.
2. Migrating Pre-v23 Machines: My feeling is we would more likely use the existing PrimeNet code and tweak it to use the new database instead of rewriting that working legacy code in new JSP/servlets - or anything else for that matter. New efforts should focus upon the new protocol & design.
3. Manual Web Forms Pages - Maybe whomever did this sweet forum system should do the forms pages! :) In v4 we have a bunch of CGI Perl scripts invoking a special pnWebForm.exe CGI app to drive this, and from there RPCs into the main system. I'd be happy to see someone run this football downfield in any language... (hey, the Superbowl is in town here this weekend!)

Nominal server load - Here's a scaling factor for our thinking: the current v4 system using 'slow' CGI sees between 2 to 3 transactions per second spiking to 10 per sec or so; it was stress tested at a rate of nearly 30x that rate continuously for several days.

Originally Posted by aga

Do computers die that often?

Every day - or nearly so. GIMPS has been in a steady state since 1999, losing machines about as fast as new ones join, about 150 per day. It's hard to know when a machine is really 'dead' since they do not send a 'death' message to the server first.

Usually machines just go MIA past a reasonable waiting period, which is presently set in v4 to 90 days of inactivity, where activity is defined as any client transaction with the server. Sometimes we assume they are dead (and expire their exponents), and then they come back online and submit a test result and continue fine, but usually they don't come back.

It would be good to know why machines 'die' and fall offline. Many - perhaps most? - are simply being decommissioned by upgrades to new PCs. Others have a change in network environment that blocks its access to the server. Some folks just quit. Check out this recent chart from the v4 server - the data is not completely correct owing to account transfers, etc., but is reasonably representative:
http://scottkurowski.com/images/pnMachineAges.jpg
There's a fairly uniform distribution of ages, except the first 4 months being risky or transfers. Not one is over 29 months old, at least as reckoned by machine ID.

[cheesehead]

(I would be quite thankful, sir, if you would kindly move that thing over to the right so it does not crimp my genuine Foamation cheesehat.)




(I would be quite thankful, sir, if you would kindly move that thing over to the right so it does not crimp my genuine Foamation cheesehat.)




(I would be quite thankful, sir, if you would kindly move that thing over to the right so it does not crimp my genuine Foamation cheesehat.)




(I would be quite thankful, sir, if you would kindly move that thing over to the right so it does not crimp my genuine Foamation cheesehat.)

Originally Posted by Old man PrimeNet

Not one is over 29 months old, at least as reckoned by machine ID.

Hunh??

My "slow" system has certainly been reporting results to Primenet longer than 29 months. I've never changed my original userid or its original machine ID, and it's been steadily submitting accepted results as recently as last week.

[Old man PrimeNet]

I updated the image, sorry. The updated chart makes a lot more sense. The oldest still-running node is, not surprisingly, my 'challenge' account's GIMPS_NODE0 at 70 months - the 200Mhz PII that was my main PC when I wrote PrimeNet in 1997.

[QuintLeo]

I wonder what the big jump in machines around the 48 month ago mark was?

[aga]

Originally Posted by Old man PrimeNet

I read that mySQL replication can stream transactions to several peer servers concurrently. Can one filter for stats reporting, and the others as failovers not filter?

Yes, that can be done; all slaves will receive the same replication stream, but each slave can be configured to ignore particular database/tables updates. Bad thing, that you can not configure on the server that 'this table content should not be replicated to the slave 2, but should be to the slaves 3 and 4' - this is a security concern.

Originally Posted by Old man PrimeNet

... Unless a corruption propagated and brought all the servers down.

Yes, this should not be ignored. Good side is that MySQL logs updates within binary log file(s); if you have an old snapshot and all the consequent update log files, you can get database snapshot corresponding to any point in time - by simply starting with the old backup, and applying the updates until desired time point. Thus what should be backed up regularly, is the update logs (they are generally compressed by bzip2 pretty well, thus disk space and even bandwidth to regularly pull the logs to remote storage should not be a problem at all, that will be just a fraction of traffic server sees).

Originally Posted by Old man PrimeNet

Passwords in stats servers: No big deal? To me, the biggest risk having more servers in the picture is we might get sloppy with our maintenance procedures or server install/change configurations. If done right, however, I don't see much concern about stats servers having login passwords mirrored and used for detailed account viewing. You are right they would need to be protected within a trusted server, though.

Well, there is no mean to prevent stats servers from gathering passwords seen; so maybe, it will be sufficient to store not plaintext passwords, but MD5 hash (or at least, send stats servers the hash instead of plain text apssword). This is slightly less secure, but will avoid realtime dependency of stats servers upon core servers.

This basically raises more general question: what the passwords are used for. Currently they are for stats viewing only. With v5, they are going to also authentificate team affiliation changes. I think it's also reasonable to host 'newsletter subscription' stuff at server, too - embedding it with client sounds a bit strange (but I well understand why it did happen that way). And probably few other things?

There are few actions I can think of that might require to keep extra information at stats servers (where we ought keeping only information that's really needed, any way). For example, email addresses. That's is not really needed to show stats. But it will be needed for the usual function of mailing a new random password instead of forgotten one.

Maybe, it will be desirable to use 2 passwords per user, one for browsing stats, and another for tweakiing different things. Let me remind about the idea I expressed while ago: use 3 kind of servers/modules, not 2. The additional server(servers, web application) is for tweaking core database (like password changes), thus stats servers are leave only with stats processing and displaying work and nothing else. I'm still not sure which structure is better for a partiuclar task; we need to describe in details all operations/actions provided for GIMPS clients prior making a decision.

Originally Posted by Old man PrimeNet

Security mechanisms cont. - I like the 32-bit key-per-assigned-exponent idea. Maybe we can do that for the clients, too ...

I still do not understand what's the benefit doing that? I.e. what does key solve, it alows the exponent to move from user to user; do we reallly need that? In my opinion, that just opens few extra possibilities to cheat.

Tho... if the exponent key will allow to avoid an extra password, it might be a worthy thing to use.

Originally Posted by Old man PrimeNet

A random 32-bit int as a secret key is just as sufficient. Why do more?

My quick math shows that to ensure probability of key compromization at level of 0.01% or below against of non-distributed attack, 33 bits of enthropy are quite sufficient. So 32 bits sound ok.

There is another approach possible: each assigned expoenent might be provided with a digital signature (with exponent value, and test number (0 for initial LL, 1 for doublecheck, -56 for fastoring up to 56 bits, etc) signed). But: the private key is used only when exponents are added to the database/marked for additional testing, thus it's possible to avoid keeping the key at all servers. And using a public key, any core server will be and to easily check if the returned exponent AND the work done match the signature (this sounds very useful). This check is just CPU-bound and cause no disk I/O, thus is very robust against DoS attacks.

(I should also mention that this does not involve using cryptographics modules within GIMPS clients - the signature is not more harming than browser cookie; thus no problems with copyrights and laws in certain countries).

Originally Posted by Old man PrimeNet

1. First, the right design will make the choice of stateless http-to-database interface, JSP, ASP, CGI, ISAPI, etc., irrelevant anyway.

You just was mentioning traffic throttling code. With java it goes very naturally - you create a class, with interface consisting of a single method like:

boolean isok (InetAddress ia);

that returns true is the request should be served or false if the requestor should be sent to the forest. The class internally uses static object(s) to store state. Now you call the method at the beginning of each JSP or Servlet:

<%
if (!Throttler.isok(request.getRemoteAddress())) {
out.println("ERROR 12345: you should go to the forest this time");
return;
}
%>

And the rest of the JSP/Servlet code is not affected by the traffic throttling feature at all. What else can be used without sacrificing performance and portability?

Originally Posted by Old man PrimeNet

Nominal server load - Here's a scaling factor for our thinking: the current v4 system using 'slow' CGI sees between 2 to 3 transactions per second spiking to 10 per sec or so; it was stress tested at a rate of nearly 30x that rate continuously for several days.

Well, this does not look much, especially if we remember that it's 2-way SMP box. But for v5 there are already at least 2 CPU-bound additional features considered, and more interesting ideas might pop up. So I'd fel safe knowing that server(s) can handle at least 1000x traffic over anticipated level.

BTW, one more CPU-bound addition: the interserver traffic is going to be highly compressable (as limited set of messages gets repeated again and again with just minor changes); together with the fact that there is no need for instant replication it would be wrong to miss the opportunity compressing the network traffic. This can be done by tunneling traffic over SSH; or piping via gzip; but I personally prefer using java.util.zip.* standard package (which is basically interface to libz) - I bet I need 30 seconds to implement transparent compression of replication traffic that way :) and then the code does not incur overhead like extra CPU context switches.

Originally Posted by Old man PrimeNet

Every day - or nearly so.

And all of them are willing to reassign the exponent (with particular checkpoint file) to someone else? No. Those deads you are talking about didn't even run the 'quit GIMPS' function of mprime/prime95.

But the graph is interesting, thank you.

[battlemaxx]

Couldn't you have sub-team queues and have members of the sub-team draw from that pool?

The "leader" of the team could manage the queue, and handle moving work around at that level. If anything spurious came through, they could handle it or re-assign it or something like that. Not every team would like that option, but the serious ones would probably be willing to police themselves a little.

[QuintLeo]

For point of reference - I figured out a week or so back that my Tiger dual-Athlon had a 50/50 chance of cracking a DES "56 bit" key in about a year - 2 years roughly to exaust the ENTIRE keyspace.

32 bit keys are NOT secure, by most current measures - though if the server limits the number of connects from a single IP to once a second or thereabouts, it should be secure enough.

[pakaran]

Originally Posted by Prime95

We should be able to identify fake LL results without requiring triple-or-more-checking. One new feature I want to add is to make sure the double-check is sent to a different userid and different team. We should also make sure that at least one LL result comes from a trusted prime95 client - this should be easy as 99% of results come from prime95/mprime executables built by me.

If the trusted client suffers from a hardware error, you'll still need a triple-check.

Nathan

[barcode]

Is this project dead?