mersenne.ca - Page 5

henryzz · 2018-04-15, 17:02

Quote:

Originally Posted by GP2

Disk storage is cheap. PrimeNet could start storing this data too.

Hosting bandwidth can be more expensive though.

James Heinrich · 2018-04-15, 17:25

I usually see it the other way -- you have to pay for additional storage, whereas often bandwidth (e.g. 250Mbps) is fixed but data transfer is uncapped.
Not that the demand for factor tables should be tremendously high, at least I don't think so.

henryzz · 2018-04-15, 19:55

Quote:

Originally Posted by James Heinrich

I usually see it the other way -- you have to pay for additional storage, whereas often bandwidth (e.g. 250Mbps) is fixed but data transfer is uncapped.
Not that the demand for factor tables should be tremendously high, at least I don't think so.

The forum has a data transfer cap. Currently it is a non issue but it was before we upgraded plan.

snme2pm1 · 2018-04-16, 11:37

I recollect that Aaron has provisioned a greater server then ever before, and anticipated it to be under-utilised.
I also recall his long ago comments about lots of spinners! Lovely!
Is not some kind of amalgamation of forces due?
I also believe that Aaron provisioned matters such that the primenet sever was a virtual machine.
As far as I am aware, James is a Linux inclined bloke, due to past head bash emoticons in relation to Widows facilities regarding database nuisances.
So would it be not decent to create a minor related linux virtual machine for James to configure?
Thereby more forces would be combined.

Uncwilly · 2018-04-16, 14:14

I am of the type that thinks that there is value to having the data and websites spread across multiple different companies resources. What if the company that host PrimeNet has a court action or bankruptcy. Or the data center has a natural disaster or disgruntled employee with plan to do as much damage as possible.

Mirroring the data and custom software that makes up GIMPS should be on at least 3 physical devices, at least 2 wide spread locations, and on different hardware platforms (with at least 2 different storage media [a couple of thumb drives in addition to the SSD/HDD's at the server].

Mersenne.ca, PrimeNet, MF, and GPU72 are all on different companies hardware. Mirroring all datasets and code on to each makes sense. Syncing once a week from the mother site should not max anyone's bandwidth.

Madpoo · 2018-04-18, 04:51

Quote:

Originally Posted by snme2pm1

I recollect that Aaron has provisioned a greater server then ever before, and anticipated it to be under-utilised.
I also recall his long ago comments about lots of spinners! Lovely!
Is not some kind of amalgamation of forces due?
I also believe that Aaron provisioned matters such that the primenet sever was a virtual machine.
As far as I am aware, James is a Linux inclined bloke, due to past head bash emoticons in relation to Widows facilities regarding database nuisances.
So would it be not decent to create a minor related linux virtual machine for James to configure?
Thereby more forces would be combined.

Reading these posts, the thought did occur to me that maybe a consolidation would be possible. The last time I updated the hardware (when the server was physically moved from Seattle to Dallas) it gained even more CPU/memory/drive space to the point where I'm able to clone Primenet entirely and run a backup copy for testing. We used it to test some of the PRP code and database changes before it went live.

Ultimately I'd defer to George and James on whether either of them would actually want to, but technically it seems possible. The colo provider does give us a good amount of bandwidth and I'm pretty sure we have plenty to spare on a monthly basic. As far as CPU goes, Primenet typically doesn't use much although sometimes the SQL queries may spike it for a bit during more complicated maintenance, but otherwise not too bad.

Quote:

Originally Posted by Uncwilly

I am of the type that thinks that there is value to having the data and websites spread across multiple different companies resources. What if the company that host PrimeNet has a court action or bankruptcy. Or the data center has a natural disaster or disgruntled employee with plan to do as much damage as possible.
...

Speaking just for Primenet, it uses a cloud backup provider with weekly full SQL backups and I think the incremental SQL backups are every 4 hours or something like that (haven't checked lately). The cloud provider I just switched to has an option to backup the entire virtual machine, and I do have the space for it on the account, so maybe I'd look at doing that as well on some quarterly basis. Even though all the data/code is backed up, it would still take some time to rebuild the VM and get it all setup and running so maybe having a full VM snapshot would be a nice-to-have.

Uncwilly · 2018-04-18, 06:03

Quote:

Originally Posted by Madpoo

Even though all the data/code is backed up, it would still take some time to rebuild the VM and get it all setup and running so maybe having a full VM snapshot would be a nice-to-have.

James and Chris have some code that should be preserved. There are some other odds and ends around, some loosely associated with GIMPS and some independent, that would be good to have duplicates (like some of Luigi's things, all of the GPU code sets, a quarterly snap shot of mlucas, etc.)

I know PrimeNet is well set up, but considering that none of the data are on the wayback machine, a well handled second back scheme (mirroring on a different company's platform) might still be wise.

LaurV · 2018-04-19, 05:49

If external storage space needed, I can offer a corner on my shelves too. If you provide a link, I can download and store what I found there (possibly periodically). 50G is not much, can keep (say) the last 4-5 generations in a 250-300G dedicated backpack...

(edit: assuming it is zipped/encrypted already, otherwise the space to store it will be lower when zipped by myself)

James Heinrich · 2018-04-19, 10:23

See post #40 for download links, ~3.5GB zipped.

snme2pm1 · 2018-04-20, 13:37

I recall sometime ago there was occasion to speak about the Bus Factor, in regard to PrimeNet.
Well I wonder how that might play into the way that people think in near discussion for mersenne.ca.
Having resources distinctly mirrored, might offer a distinct advantage in safety terms, and I would prefer that James doesn't fall under a bus any time soon (they exhibit too many tons).

kladner · 2018-04-20, 14:39

My first thought was to wonder which (electronic) bus was in question. I realize now that it is the Bus of Unexpected Fatality.

2018-04-15, 17:25	#46
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 2×3×5×109 Posts	I usually see it the other way -- you have to pay for additional storage, whereas often bandwidth (e.g. 250Mbps) is fixed but data transfer is uncapped. Not that the demand for factor tables should be tremendously high, at least I don't think so. Last fiddled with by James Heinrich on 2018-04-15 at 17:26

2018-04-16, 11:37	#48
snme2pm1 "Graham uses ISO 8601" Mar 2014 AU, Sydney 241 Posts	I recollect that Aaron has provisioned a greater server then ever before, and anticipated it to be under-utilised. I also recall his long ago comments about lots of spinners! Lovely! Is not some kind of amalgamation of forces due? I also believe that Aaron provisioned matters such that the primenet sever was a virtual machine. As far as I am aware, James is a Linux inclined bloke, due to past head bash emoticons in relation to Widows facilities regarding database nuisances. So would it be not decent to create a minor related linux virtual machine for James to configure? Thereby more forces would be combined. Last fiddled with by snme2pm1 on 2018-04-16 at 11:38

2018-04-19, 05:49	#52
LaurV Romulan Interpreter Jun 2011 Thailand 5²·7·53 Posts	If external storage space needed, I can offer a corner on my shelves too. If you provide a link, I can download and store what I found there (possibly periodically). 50G is not much, can keep (say) the last 4-5 generations in a 250-300G dedicated backpack... (edit: assuming it is zipped/encrypted already, otherwise the space to store it will be lower when zipped by myself) Last fiddled with by LaurV on 2018-04-19 at 05:51

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Small inconsistencies between mersenne.org and mersenne.ca factor databases	GP2	mersenne.ca	44	2016-06-19 19:29
mersenne.ca (ex mersenne-aries.sili.net)	LaurV	mersenne.ca	8	2013-11-25 21:01
Gaussian-Mersenne & Eisenstein-Mersenne primes	siegert81	Math	2	2011-09-19 17:36
Mersenne Wiki: Improving the mersenne primes web site by FOSS methods	optim	PrimeNet	13	2004-07-09 13:51

2018-04-16, 14:14	#49
Uncwilly 6809 > 6502 """"""""""""""""""" Aug 2003 101×103 Posts 2²·3·11·71 Posts	I am of the type that thinks that there is value to having the data and websites spread across multiple different companies resources. What if the company that host PrimeNet has a court action or bankruptcy. Or the data center has a natural disaster or disgruntled employee with plan to do as much damage as possible. Mirroring the data and custom software that makes up GIMPS should be on at least 3 physical devices, at least 2 wide spread locations, and on different hardware platforms (with at least 2 different storage media [a couple of thumb drives in addition to the SSD/HDD's at the server]. Mersenne.ca, PrimeNet, MF, and GPU72 are all on different companies hardware. Mirroring all datasets and code on to each makes sense. Syncing once a week from the mother site should not max anyone's bandwidth.

2018-04-19, 10:23	#53
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 110011000110₂ Posts	See post #40 for download links, ~3.5GB zipped.

2018-04-20, 13:37	#54
snme2pm1 "Graham uses ISO 8601" Mar 2014 AU, Sydney 241 Posts	I recall sometime ago there was occasion to speak about the Bus Factor, in regard to PrimeNet. Well I wonder how that might play into the way that people think in near discussion for mersenne.ca. Having resources distinctly mirrored, might offer a distinct advantage in safety terms, and I would prefer that James doesn't fall under a bus any time soon (they exhibit too many tons).

2018-04-20, 14:39	#55
kladner "Kieren" Jul 2011 In My Own Galaxy! 2²×2,539 Posts	My first thought was to wonder which (electronic) bus was in question. I realize now that it is the Bus of Unexpected Fatality.