mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Software

Reply
 
Thread Tools
Old 2018-09-05, 13:23   #1
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

26508 Posts
Default Google compute engine

I has one problem: on machine with 8 vCPU, on first thread I only can get 4 cores per test not 5 as written in file.
Second worker works with 3 cores without any problem



[Worker #1]
CoresPerTest=5


[Worker #2]
CoresPerTest=3

Last fiddled with by pepi37 on 2018-09-05 at 13:54 Reason: add more info
pepi37 is offline   Reply With Quote
Old 2018-09-05, 13:47   #2
GP2
 
GP2's Avatar
 
Sep 2003

50318 Posts
Default

On both AWS and GCE, eight vCPU actually means four cores with hyperthreading.

So your total CoresPerTest should add up to 4, not 8.


Also,
have you benchmarked mprime on GCE vs. mprime on AWS EC2? If I recall correctly the Skylakes on GCE are considerably slower than on AWS, and to make matters worse, you have to be careful to specify Skylake or better on GCE, otherwise you might get some other architecture. Make sure your /proc/cpuinfo output contains avx512 elements in the "flags" line.

On the other hand GCE has a fixed cost of 1.5 cents/hour per core for preemptible instances while AWS spot prices are variable and currently around 1.9 cents/hour for a one-core instance, so that would compensate for lesser throughput. I forget exactly where the breakeven point is, though.
GP2 is offline   Reply With Quote
Old 2018-09-05, 13:52   #3
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

23×181 Posts
Default

GP2, thanks for reply, but I manage to get 99% utilization with somehow weird setup
Yes, I choose Skylake, and find cheapest region and lower memory requirements as low as I can: so with free 300$ I can manage about 70 days of work.
setup is next


Affinity=99
NumCPUs=5

ConfigureHyperthreads=1
CpuNumHyperthreads=2
WorkerThreads=2
Pid=0
OldCpuSpeed=2504
NewCpuSpeedCount=0
NewCpuSpeed=0

[Worker #1]
CoresPerTest=5


[Worker #2]
CoresPerTest=3
mprime works without any warning , and utilization is 97%

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) CPU @ 2.00GHz
Stepping: 3
CPU MHz: 2000.172
BogoMIPS: 4000.34
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 56320K
NUMA node0 CPU(s): 0-7
Flags: ----- avx512cd avx512bw avx512vl
fma3 is detected in mprime

Last fiddled with by pepi37 on 2018-09-05 at 14:05 Reason: add more info
pepi37 is offline   Reply With Quote
Old 2018-09-06, 04:20   #4
GP2
 
GP2's Avatar
 
Sep 2003

258510 Posts
Default

I don't think it ever makes sense to specify more cores in your mprime configuration file than the actual physical cores available. You might get "99% utilization", but that doesn't mean it's performing efficiently.

In any case, you are better off running four instances of a one-core (2 vCPU) virtual machine (n1-highcpu-2) rather than one instance of a four-core (8 vCPU) virtual machine. On GCE the cost for this will be exactly the same, but your total throughput will be considerably higher.

It's easy to "clone" (Google's terminology) your existing setup to create new virtual machines. Make sure the new ones also use Skylake though.

On GCE the pricing is mostly uniform, the only difference is that the non-US-based servers (Europe, Asia, Canada) are about 10% more expensive than the ones in the US, so always use a US server.

I hope you are using preemptible instances, they're much, much cheaper. It's annoying because you have to restart them at least once a day, and sometimes even more often than that. You can get the CloudConsole app for your phone, it's convenient for doing the daily restarts.
GP2 is offline   Reply With Quote
Old 2018-09-06, 12:02   #5
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

23×181 Posts
Default

Quote:
Originally Posted by GP2 View Post
.but your total throughput will be considerably higher.

It's easy to "clone" (Google's terminology) your existing setup to create new virtual machines. Make sure the new ones also use Skylake though.

Small test show me increase in speed about 15% if I use 4 cpu2 instance instead 1 cpu8 instance. Thanks for pointing: but since it is free, and since I would like to setup, run and forget, I choose one instance 8vcpu.
Also I would not give to myself some extra job to start and stop instances on daily bases :)
I total : this instance will give me ( in predicted time of free$) about 1400 workunits from 1.5 to 2.2 milion digits: so it is not bad for free thing!
pepi37 is offline   Reply With Quote
Old 2018-09-06, 21:08   #6
GP2
 
GP2's Avatar
 
Sep 2003

258510 Posts
Default

Quote:
Originally Posted by pepi37 View Post
Small test show me increase in speed about 15% if I use 4 cpu2 instance instead 1 cpu8 instance. Thanks for pointing: but since it is free, and since I would like to setup, run and forget, I choose one instance 8vcpu.
Also I would not give to myself some extra job to start and stop instances on daily bases :)
I total : this instance will give me ( in predicted time of free$) about 1400 workunits from 1.5 to 2.2 milion digits: so it is not bad for free thing!
Preemptible instances cost 1.5 cents an hour, the regular instances cost 7.09 cents an hour. That's an 80% saving!!

If you're running a website or other Internet application, it has to keep running without interruption. But what we are doing is essentially a batch job. There is no uptime requirement, no deadline. It doesn't matter if it's down for part of the day.

What's your goal? If you're trying to discover a Mersenne prime with that free $300, you can easily quintuple your chances with very little extra effort, just by using preemptible instances and multiple one-core instances.


PS,
GCE lets you specify a startup script. So you can automatically restart mprime whenever the preemptible instance restarts. You literally only need to download the CloudConsole app onto your phone, and check it once a day and restart your preemptible instances as necessary. A few seconds of effort each day, no more than that. You never need to log in to a shell terminal.

Last fiddled with by GP2 on 2018-09-06 at 21:17
GP2 is offline   Reply With Quote
Old 2018-09-06, 22:41   #7
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

23×181 Posts
Default

Quote:
Originally Posted by GP2 View Post

What's your goal? If you're trying to discover a Mersenne prime with that free $300, you can easily quintuple your chances with very little extra effort, just by using preemptible instances and multiple one-core instances.

This is not my goal, I have much less limits. One prime with 2M digits if good for me!

But your point is clear ( thanks for that - I just now stop machine and build new preemptible instance) 6x times longer means many more processed tasks
P.S

Must find place where to put script for restart mprime

Last fiddled with by pepi37 on 2018-09-06 at 22:42
pepi37 is offline   Reply With Quote
Old 2018-09-07, 01:36   #8
GP2
 
GP2's Avatar
 
Sep 2003

5×11×47 Posts
Default

Quote:
Originally Posted by pepi37 View Post
Must find place where to put script for restart mprime
Go to the Compute Engine dashboard, on the page that lists your VM instances.

Click on one of those instances and you'll get to a page where you scroll down until you reach "Custom metadata" and "shutdown-script" and "startup-script". Edit that and just start up mprime in there.
GP2 is offline   Reply With Quote
Old 2018-09-07, 03:03   #9
retina
Undefined
 
retina's Avatar
 
"The unspeakable one"
Jun 2006
My evil lair

141048 Posts
Default

Quote:
Originally Posted by GP2 View Post
...and check it once a day and restart your preemptible instances as necessary.
Yeah, this "check it once per day" thing is just awful IMO. The cost of free computing is just too high.

It's like those terrible credit cards. Check it once per day to make sure no one has stolen your money. Aaargh, no thanks. Too much hassle for too little gain.
retina is online now   Reply With Quote
Old 2018-09-07, 07:37   #10
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

23×181 Posts
Default

Quote:
Originally Posted by retina View Post
Yeah, this "check it once per day" thing is just awful IMO. The cost of free computing is just too high.

It's like those terrible credit cards. Check it once per day to make sure no one has stolen your money. Aaargh, no thanks. Too much hassle for too little gain.

Please just not fight between


In our world on this small blue dot, very rare things are free. When I discovered this : quick calculation give me 60 days of 24/7 processing :I stay on one instance and "power" of that 8vCPU instance is nearly same as my I5-3750 K at 3.4 Ghz. Yes, I specifically select Skylake or newer, but those CPU work at 2 GHz, and I5-3570K works at 3.5 Ghz.So they are fast, since on 2 Ghz have same speed as for 3.5 Ghz.
But when I get option to get 6x more time on same machine, I know it must be some catch (22) : And yes, instance will be stopped if work 24 hour or less.

Download mobile app , and as GP2 say: just start it , and enjoy.
Since we here are not on the race , in fact I lost nothing, and got much more.
So at the end I am very pleased with new situation: and 6 x more compute "free time" :)


P.S Too much hassle for too little gain
I will not agree with that: using 8 instances with 1 vCPu will give me gain about 15 % compared to one 8vCPu instance: and that is "little gain" ,but get 6 times more time for same price is not little gain.In one case I can process about 1000 ( my tasks) and on the other 6000 tasks. I thing difference is ( very) noticeable :)

Last fiddled with by pepi37 on 2018-09-07 at 07:42 Reason: add more text
pepi37 is offline   Reply With Quote
Old 2018-09-07, 07:46   #11
ET_
Banned
 
ET_'s Avatar
 
"Luigi"
Aug 2002
Team Italia

24×7×43 Posts
Default

Quote:
Originally Posted by pepi37 View Post
Please just not fight between


In our world on this small blue dot, very rare things are free. When I discovered this : quick calculation give me 60 days of 24/7 processing :I stay on one instance and "power" of that 8vCPU instance is nearly same as my I5-3750 K at 3.4 Ghz. Yes, I specifically select Skylake or newer, but those CPU work at 2 GHz, and I5-3570K works at 3.5 Ghz.So they are fast, since on 2 Ghz have same speed as for 3.5 Ghz.
But when I get option to get 6x more time on same machine, I know it must be some catch (22) : And yes, instance will be stopped if work 24 hour or less.

Download mobile app , and as GP2 say: just start it , and enjoy.
Since we here are not on the race , in fact I lost nothing, and got much more.
So at the end I am very pleased with new situation: and 6 x more compute "free time" :)


P.S Too much hassle for too little gain
I will not agree with that: using 8 instances with 1 vCPu will give me gain about 15 % compared to one 8vCPu instance: and that is "little gain" ,but get 6 times more time for same price is not little gain.In one case I can process about 1000 ( my tasks) and on the other 6000 tasks. I thing difference is ( very) noticeable :)
BTW, how did you get that $300 bonus?
ET_ is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
How-to guide for running LL tests on Google Compute Engine cloud GP2 Cloud Computing 4 2020-08-03 11:21
Google Compute Engine GP2 Cloud Computing 32 2018-01-23 02:16
New GPU Compute System airsquirrels GPU Computing 90 2017-12-08 00:13
New Compute Box Christenson Hardware 0 2011-01-15 04:44
My throughput does not compute... petrw1 Hardware 9 2007-08-13 14:38

All times are UTC. The time now is 02:32.


Tue Jul 27 02:32:17 UTC 2021 up 3 days, 21:01, 0 users, load averages: 1.61, 1.54, 1.55

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.