mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > Cloud Computing

Reply
 
Thread Tools
Old 2016-12-06, 22:37   #1
GP2
 
GP2's Avatar
 
Sep 2003

29·89 Posts
Default Batch jobs using AWS Batch

In the old days, looking for Mersenne primes meant writing your program on punch cards and submitting a batch job. The next day, or whenever, you'd go pick up your printout and find out that M4423 and M4253 are prime. These days computers have an interactive interface, but you don't really need that interactivity at all for the kind of number crunching we do.

Apparently Amazon is reviving the batch job concept by introducing AWS Batch. It's still in preview, so information is limited, but it sounds like you just specify how much memory you need and how many cores, and the rest is handled automatically.

Currently, there's often a mismatch between what virtual machine instances offer and what a calculation needs. LL testing needs as many CPUs as possible but very little memory; an ambitious P−1 or ECM stage 2 calculation might need the exact opposite. You can mix and match different types of work to try to get the optimal use out of an expensive instance with multiple cores and lots of memory, but it's easier if Amazon handles all that for you by scheduling everyone's big and small jobs in some optimal way, so you never have to worry about starting and stopping cloud instances yourself. By using resources more efficiently and being flexible about the completion time, perhaps the price might be lower too.

It's hard to say, since signing up for the preview might require a non-disclosure agreement, but it looks promising. Among other things, it might become trivial to purchase, say, a dozen LL tests in some defined range for some fixed price.
GP2 is offline   Reply With Quote
Old 2016-12-07, 11:31   #2
ET_
Banned
 
ET_'s Avatar
 
"Luigi"
Aug 2002
Team Italia

25·149 Posts
Default

Quote:
Originally Posted by GP2 View Post
In the old days, looking for Mersenne primes meant writing your program on punch cards and submitting a batch job. The next day, or whenever, you'd go pick up your printout and find out that M4423 and M4253 are prime. These days computers have an interactive interface, but you don't really need that interactivity at all for the kind of number crunching we do.

Apparently Amazon is reviving the batch job concept by introducing AWS Batch. It's still in preview, so information is limited, but it sounds like you just specify how much memory you need and how many cores, and the rest is handled automatically.

Currently, there's often a mismatch between what virtual machine instances offer and what a calculation needs. LL testing needs as many CPUs as possible but very little memory; an ambitious P−1 or ECM stage 2 calculation might need the exact opposite. You can mix and match different types of work to try to get the optimal use out of an expensive instance with multiple cores and lots of memory, but it's easier if Amazon handles all that for you by scheduling everyone's big and small jobs in some optimal way, so you never have to worry about starting and stopping cloud instances yourself. By using resources more efficiently and being flexible about the completion time, perhaps the price might be lower too.

It's hard to say, since signing up for the preview might require a non-disclosure agreement, but it looks promising. Among other things, it might become trivial to purchase, say, a dozen LL tests in some defined range for some fixed price.
Back to JCL then...
I suppose there will be such descriptive langage to take full advantage of the system...
ET_ is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
15e batch of WU's NFS@Home pinhodecarlos NFS@Home 31 2020-01-21 21:49
Batch File Issue, Perhaps storm5510 Programming 3 2018-01-18 04:44
14e batch of WU's pinhodecarlos NFS@Home 25 2015-07-25 17:50
Got first batch of TF to 75 Chuck GPU to 72 23 2015-05-10 20:34
batch file em99010pepe No Prime Left Behind 7 2008-03-15 10:53

All times are UTC. The time now is 09:20.

Fri Oct 23 09:20:41 UTC 2020 up 43 days, 6:31, 0 users, load averages: 1.34, 1.49, 1.63

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.