Replacing Entropia's primenet server

[Prime95]

With the many server outages the last three weeks the question has been raised, why not have our own GIMPS server?

First off, some background. Scott Kurowski founded Entropia as a for-profit startup in the field of distributed computing. GIMPS was, in essence, his proof of concept to investors. In 1997, Primenet was launched and steadily improved in 1998. All this was done at no cost to GIMPS and they have been gracious enough to continue supporting Primenet ever since. Server availability has been pretty good during that time. Roughly one week a year we have an irritating outage that takes time to track down and resolve. Despite the recent outage GIMPSers have been quite appreciative of all Entropia has done for us rather than flaming about the lousy service.

The Primenet server is on an NT machine. The database is SQLServer. Entropia owns all the server source code.

The current situation has several disadvantages:

1) If Entropia folds or has a change in leadership, they may no longer support Primenet. Scott has assured me has the legal right and would be able to get a new primenet server up and running in short order.

2) I feel guilty asking a for-profit company that has been good to us to devote development dollars on new features that would improve the GIMPS experience. This includes teams with members having their own userids, P-1 as a separate work type, better charts and graphs, better synchronization or even better merging the master and primenet database, etc.

3) Brad is my point-of-contact. If he is out of town, it can be difficult getting him to nudge the proper folks to fix a primenet problem.

4) The server is 3000 miles away from me. Many of these problems wouldn't exist if we were in the same town.


What would it take to run our own server?

1) A machine with enough bandwidth and proper up-time guarantee.

2) Money to run such a machine.

3) All server side software needs to be rewritten and improved and tested.

4) Existing accounts and stats need to be migrated to the new database.

I did some preliminary investigative work last year. Linux / Apache / PostgreSQL / PHP seems to provide all the basic tools. I guess MySQL would work as I see they now support Commit / Rollback.

This would be a substantial development effort. How have other distributed projects handled this? Would a community development effort be feasible? I'm sure there are some issues I've overlooked and welcome any comments on the subject.

[Prime95]

One more point - security. Any new server would need monitoring for DOS attacks, attempted or successful hackings, needed software upgrades, etc. I am not a cyber-security expert. My background is actually database programming.

[xtreme2k]

How much bandwidth does the existing server use? What kind of machine is it currently being run on?

Colo cost and bandwidth are not cheap with Tier 1 providers. Plus the cost of the machine itself. The only main problem is cost I believe. Where would the money be generated from?

Porting from NT to Linux isnt an easy task. Managing it is also totally different.

For DOS attacks there really isnt much you can do on the server itself other than not responding to the individual packets, the provider needs to help out and block some of the traffic upstreams.

[toferc]

Over the years, the Primenet server has proven to be both robust and stable. Reimplementing its functionality and maintaining a server are nontrivial tasks, to say the least. I'd like to know if there is anything that we can do to help keep it running, without reinventing the wheel. Unfortunately, I don't even know what direction to think in since there's so much I don't know. So... here are a few questions for you:

Are the issues related to failing hardware? If some new server hardware were to fall from heaven, would moving the software to a new machine fix the problem? If so, is moving the software even a realistic option? Would it be too labor intensive for Entropia to do for free?

Is the problem related to excessive CPU/IO utilization? If a hardware upgrade is not a good answer (see above), would it be possible for volunteers to attempt to make speed improvements? I don't hold out much hope for this one due to IP issues and difficulty in simulating performance without access to the actual server, but it's worth asking whether tuning a few queries might alleviate the problem. Incidentally, I've often had an itch to see the code that seems to take so long to generate that status report. ;)

Are the WWW pages being served by the same machine? Are they a significant resource drain? Would relocating this workload to a new server help alleviate things? Individual account reports probably could not be moved, but snapshot stats and anything static could be. This would also facilitate the maintenance/improvement of purely WWW content such as pretty charts, graphs, and other derivative statistics.

Admittedly, these are all band-aids and leave some core issues (new functionality, dependence on proprietary software) unaddressed, but might they be worthwhile?

Tofer

[Prime95]

Originally Posted by xtreme2k

How much bandwidth does the existing server use? What kind of machine is it currently being run on?

a dual 800Mhz PIII with 36GB fast RAID 5 disks (18GB usable) and 256MB, on a T1 or DS3 (not sure which) with Win2000.

[binarydigits]

I hope I didn't come across as harsh in the other thread. I was already familiar with the background and I appreciate everything Entropia has done for GIMPS. It would be difficult to set up a new system which would make an overall improvement in what they are already doing. It would probably be a full-time job for someone to keep it running smoothly (or at least full-time on call).

If we want to try, I believe a community development effort would be feasible. Apparently a lot of people are willing to devote time &/or resources. But once it is developed it cannot be run "by committee". Also, besides startup costs there would be ongoing costs and we would need to ensure that George doesn't get stuck with those down the road.

One thing I am curious about: would Scott be hurt, or relieved, if we were to move?

[AP]

Hi all,

please do not forget that GIMPS has done as much for Entropia as they did for us, to say the least. What do you need when starting a company which wants to sell dc-power?

Yes, you need at least one running project. And this is GIMPS. So where would Entropia be without GIMPS....?

And I do not think that the problems started Dec 2001. I joined GIMPS Dec 1998 and there were more outages during that time. A friend of mine got a new P4 but if you use the manual way to get exponents from the server, it doesn´t know this type of processor.

"Primenet news and information" is more than one year old and talking about V19 of PRIME95.

Statistics charts..... March 2001..........

Regards
Achim

[Xyzzy]

This server has MySQL and PHP and stuff, and is rated for 500MB a day... I don't know if that is enough, and I don't know what CPU load is required to run PrimeNet... The bandwidth here can be expanded...

That said, hosting the static web pages would be trivial... I have no clue about the database...

I do have the capability of colocating a server for 100 bux a month... The hardware would cost around 1000 bux... But it would be Solaris based as this is all I know... The connection would be 1 megabit sustained... I'd be willing to donate a few hundred bux towards a solution like this...

I can also host a box here at home for around the same cost... If you can get Brad to mail me the box I am willing to just plug it in if you want... It would be a dedicated 768Kb/s SDSL line...

If someone needs a MySQL database to goof around with to try to fix this just let me know...

Personally, the longer I run GIMPS the less I find a need for PrimeNet... Sure the stats are fun, but George keeps the master database anyways... My concern now isn't about getting points, it is just making sure I don't replicate work... I'm not saying PrimeNet is worthless... I'm just saying that I don't need it... This is just my personal opinion, so please don't take it the wrong way... :)

[guido72]

Originally Posted by Xyzzy

Personally, the longer I run GIMPS the less I find a need for PrimeNet... Sure the stats are fun, but George keeps the master database anyways... My concern now isn't about getting points, it is just making sure I don't replicate work... I'm not saying PrimeNet is worthless... I'm just saying that I don't need it... This is just my personal opinion, so please don't take it the wrong way... :)

Hi Mike! Unfortunately it seems to me that if we really want to attract initiates on our own project, we have to arrange a nice packaging, like a gift wrapping... Some interesting stats page or an updated status page, as Akim said, may help, I think. Think about all the graphics versions of some clients: you'll never be able to convince all the people to turn-down their screen savers 'cause they'r "stealing" cpu cycles we may use for other purposes... So no way but to give'em an inefficient client which is nevertheless better than nothing... Even more this is a math project, so it is aiming at ethereal and impalpable stuff like a strange kind of numbers, not like Seti, just for an example ;)

[ebx]

Well, lets put together a plan and set up a CVS.

I am scared whenever I heard a mission critical server ran on MS Windows of any kind. It is good we can move away from there. Even if we dont change the hosting, switch to linux could save us a lot of outage, I suppose.

Do we get to access the currnet Windows server code and/or documents?

Startup costs is less of a concern. Those very capable rack mount dual AMD servers go for under 3 grands(we wont run prime95 on the server, will we?). If we dont generate results on the fly, we dont even need that much CPU power. The hosting, however, costs more for the long run. 500MB daily isnt enough. T1 should do.

And we are going to hit that 10M digit prime tomorrow. That could fund us 20 times. :D

[Prime95]

Originally Posted by Xyzzy

I do have the capability of colocating a server for 100 bux a month... The hardware would cost around 1000 bux

We're still in the early investigation stage. What would a home T1 line cost? Would a 100 bux/month colocating service agreement provide enough bandwidth and service? What are the downsides to colocation? What happens if their are hardware troubles on the colocated server? or if there is a botched software upgrade? Someone mentions using a home DSL line, what are the disadvantages of that?

Finally, if we decide to do this it won't happen quickly, at least not until I move to my new house Jan. or Feb. next year.