Big downloads?

Xyzzy · 2012-06-09, 19:11

For quite a while we maintained this page:

http://www.mersennewiki.org/index.ph...ersenne_primes

But, having a lot of people (or maybe one repeatedly?) downloading 47.txt really used up our bandwidth allowance, so we put a block on the downloads.

We are curious how we could fix this.

Perhaps a big torrent (around 252MiB uncompressed) with all of the numbers?
Perhaps a bunch of little torrents?
Perhaps a free download service, like Dropbox? (We figure there is some sort of limit, though.)

We think if just ordinary (curious) people downloaded the files the bandwidth would be reasonable, but it has been a problem in the past when a new prime is announced or when someone decides to deplete our bandwith allowance.

Suggestions?

(FWIW, we are willing to seed a torrent or multiple torrents but we would need several other people to have the download speed be reasonable and to provide redundancy.)

Thanks!

Batalov · 2012-06-09, 19:16

I think it would be best to replace it with (the code and) executables that could be downloaded and will produce this number on the people's computer. It could be even improved with being able to do it in any given base. (Maybe some Babylonian followers would want to see the M47 in base 60. And they can!)

P.S. You can also simply edit those links into e.g. http://prime.isthe.com/chongo/tech/m...9/prime-c.html
(and all others are in http://www.isthe.com/chongo/tech/mat.../mersenne.html )

chalsall · 2012-06-09, 19:44

Quote:

Originally Posted by Xyzzy

Suggestions?

The server GPU72.com is on has a monthly transmission allotment of 2 TB a month.

I would be happy to set up a virtual host (say, "files.mersennewiki.org") on the server and give you (Mike) FTP access to the root of that server. You would simply have to set up a DNS entry for the domain pointing to an IP I would give you, upload the files, and change the links.

Let me know (via PM) if you're interested in this solution.

Xyzzy · 2012-06-09, 20:01

PM sent.

Batalov · 2012-06-09, 21:09

At the very least they should be (g)zipped?
Also, the webserver can be instructed to serve "47.txt" as a stream from "unzip -p 47.zip". (But then again, this will defeat the attempt to save traffic.)

What's wrong with using Noll's links? (just curious)
It's not like someone owns the M47 or its decimal representation. And he is already dealing with the hassle of serving these files. This is a Wiki page, right?

Would you like to host the human genome sequence? I wouldn't; I'd give a link to UCSC or NCBI.

Xyzzy · 2012-06-09, 21:30

Problem solved. Chalsall is awesome. That is all.

click here

Xyzzy · 2012-06-09, 21:33

Quote:

What's wrong with using Noll's links? (just curious)
It's not like someone owns the M47 or its decimal representation. And he is already dealing with the hassle of serving these files. This is a Wiki page, right?

We think we didn't know about his links when we started and it seemed easier to keep things the way they were, or something. A lot of what we do involves very little thinking.

Dubslow · 2012-06-09, 22:15

[ASIDE]
Someone should remove the asterisk from M41, and change "between the 40th () and the 47th () on" to "between the 41th () and the 47th () on".

[/ASIDE]

chalsall · 2012-06-09, 22:16

Quote:

Originally Posted by Xyzzy

Problem solved.

Very kind Mike. But this was easy to do, and I've got bandwidth to burn.

Quote:

Originally Posted by Batalov

At the very least they should be (g)zipped.

Good point.

Three lines in the .htaccess file, and one run of "find", and the server will now send the data in GZIP format if the browser says it can handle it.

jasonp · 2012-06-10, 00:26

Um, the incident Mike is talking about involved some joker in Europe downloading the files tens of thousands of times from a list of constantly updated proxied IPs. chalsall, this sounds like it's putting you in harm's way...

chalsall · 2012-06-10, 01:19

Quote:

Originally Posted by jasonp

chalsall, this sounds like it's putting you in harm's way...

Appreciate the warning.

I've put on an hourly quota.

2012-06-09, 19:11	#1
Xyzzy Aug 2002 2×5²×13² Posts	Big downloads? For quite a while we maintained this page: http://www.mersennewiki.org/index.ph...ersenne_primes But, having a lot of people (or maybe one repeatedly?) downloading 47.txt really used up our bandwidth allowance, so we put a block on the downloads. We are curious how we could fix this. Perhaps a big torrent (around 252MiB uncompressed) with all of the numbers? Perhaps a bunch of little torrents? Perhaps a free download service, like Dropbox? (We figure there is some sort of limit, though.) We think if just ordinary (curious) people downloaded the files the bandwidth would be reasonable, but it has been a problem in the past when a new prime is announced or when someone decides to deplete our bandwith allowance. Suggestions? (FWIW, we are willing to seed a torrent or multiple torrents but we would need several other people to have the download speed be reasonable and to provide redundancy.) Thanks!

2012-06-09, 19:16	#2
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 265C₁₆ Posts	I think it would be best to replace it with (the code and) executables that could be downloaded and will produce this number on the people's computer. It could be even improved with being able to do it in any given base. (Maybe some Babylonian followers would want to see the M47 in base 60. And they can!) P.S. You can also simply edit those links into e.g. http://prime.isthe.com/chongo/tech/m...9/prime-c.html (and all others are in http://www.isthe.com/chongo/tech/mat.../mersenne.html ) Last fiddled with by Batalov on 2012-06-09 at 19:24

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
http://www.nfsnet.org/downloads/nfsnet-04145-powerpc-MacOSX.tgz	Death	NFSNET Discussion	15	2004-06-22 07:35

2012-06-09, 20:01	#4
Xyzzy Aug 2002 2×5²×13² Posts	PM sent.

2012-06-09, 21:09	#5
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 2²·5·491 Posts	At the very least they should be (g)zipped? Also, the webserver can be instructed to serve "47.txt" as a stream from "unzip -p 47.zip". (But then again, this will defeat the attempt to save traffic.) What's wrong with using Noll's links? (just curious) It's not like someone owns the M47 or its decimal representation. And he is already dealing with the hassle of serving these files. This is a Wiki page, right? Would you like to host the human genome sequence? I wouldn't; I'd give a link to UCSC or NCBI.

2012-06-09, 21:30	#6
Xyzzy Aug 2002 2·5²·13² Posts	Problem solved. Chalsall is awesome. That is all. click here

2012-06-09, 22:15	#8
Dubslow Basketry That Evening! "Bunslow the Bold" Jun 2011 40<A<43 -89<O<-88 1110000110101₂ Posts	[ASIDE] Someone should remove the asterisk from M41, and change "between the 40th () and the 47th () on" to "between the 41th () and the 47th () on". [/ASIDE]

2012-06-10, 00:26	#10
jasonp Tribal Bullet Oct 2004 5×709 Posts	Um, the incident Mike is talking about involved some joker in Europe downloading the files tens of thousands of times from a list of constantly updated proxied IPs. chalsall, this sounds like it's putting you in harm's way...