18 core Haswell/P-1 CPU load

[nucleon]

Thought this screenshot may be of interest. It's a screenshot of htop under stage2 of P-1 exponent. It's a c4.8xlarge instance on AWS EC2. Intel Xeon E5-2666 v3.

Memory starvation anyone?

I'll have to play around, to see what's best utilization of this hardware.

[aurashift]

Originally Posted by nucleon

Thought this screenshot may be of interest. It's a screenshot of htop under stage2 of P-1 exponent. It's a c4.8xlarge instance on AWS EC2. Intel Xeon E5-2666 v3.

Memory starvation anyone?

I'll have to play around, to see what's best utilization of this hardware.

I'm not familiar with AWS. This is running on dedicated hardware with dual or quad channel RAM?

[nucleon]

Blackbox.

Presumably given the size of RAM given to us, I think we're good to assume quad channel.

Public information on instance types:

https://aws.amazon.com/ec2/instance-types/

[Mark Rose]

A c4.8xlarge is a dual CPU machine.

If you get a c4.4xlarge or smaller, the virtual cores will all be allocated on a single CPU. I asked Amazon about this.

Each virtual core in the current generation is a hyperthreaded core. I don't know if you get both halves of the same physical core or not, but I would imagine so, so other tenants on the same hardware would get consistent performance.

I haven't asked if they use quad channel or not, but it would be very strange if they didn't.

[aurashift]

Does EC2 run VMWare? It'll schedule everything on the physical cores first before going for logical. (which is true for everything...I think)

[Mark Rose]

Originally Posted by aurashift

Does EC2 run VMWare? It'll schedule everything on the physical cores first before going for logical. (which is true for everything...I think)

EC2 is Xen under the hood.

[nucleon]

I chose the c4.8xlarge mainly for the amount of ram given (60GB).

Performance varies. In my experimentation, you're basically existing on the same physical box as another VM. If other VMs hosted on the same box are memory bandwidth conservative - good win for you. If not, you can see performance drop.

So my mucking around this evening, I was getting good results with 2x workers: (1) P-1 testing (2) double check. As the double check doesn't need much, I can allocate the vast bulk of ram to the P-1 process.

The difference in performance between 8cores and 16cores was minimal.

Results below.

#16cores; 1x double check
[Work thread Apr 25 11:05] Iteration: 290000 / 38961211 [0.74%], ms/iter:  1.228, ETA: 13:11:47
[Work thread Apr 25 11:05] Iteration: 300000 / 38961211 [0.76%], ms/iter:  1.228, ETA: 13:11:33
[Work thread Apr 25 11:05] Iteration: 310000 / 38961211 [0.79%], ms/iter:  1.227, ETA: 13:10:28

#1core; 1x double check
[Work thread Apr 25 11:07] Iteration: 330000 / 38961211 [0.84%], ms/iter:  9.567, ETA: 4d 06:39
[Work thread Apr 25 11:09] Iteration: 340000 / 38961211 [0.87%], ms/iter:  9.412, ETA: 4d 04:58
[Work thread Apr 25 11:10] Iteration: 350000 / 38961211 [0.89%], ms/iter:  9.425, ETA: 4d 05:04

#8cores; 1x double check
[Work thread Apr 25 11:13] Iteration: 370000 / 38961211 [0.94%], ms/iter:  1.492, ETA: 15:59:55
[Work thread Apr 25 11:13] Iteration: 380000 / 38961211 [0.97%], ms/iter:  1.496, ETA: 16:01:44
[Work thread Apr 25 11:14] Iteration: 390000 / 38961211 [1.00%], ms/iter:  1.500, ETA: 16:04:29

#8cores x2 workers; 2x double checks
[Worker #1 Apr 25 11:19] Iteration: 510000 / 38961211 [1.30%], ms/iter:  1.502, ETA: 16:02:50
[Worker #2 Apr 25 11:19] Iteration: 80000 / 38967377 [0.20%], ms/iter:  1.432, ETA: 15:27:54
[Worker #1 Apr 25 11:19] Iteration: 520000 / 38961211 [1.33%], ms/iter:  1.504, ETA: 16:03:19
[Worker #2 Apr 25 11:19] Iteration: 90000 / 38967377 [0.23%], ms/iter:  1.432, ETA: 15:27:38
[Worker #1 Apr 25 11:19] Iteration: 530000 / 38961211 [1.36%], ms/iter:  1.500, ETA: 16:00:52
[Worker #2 Apr 25 11:19] Iteration: 100000 / 38967377 [0.25%], ms/iter:  1.430, ETA: 15:26:30

[Mark Rose]

Originally Posted by nucleon

I chose the c4.8xlarge mainly for the amount of ram given (60GB).

I would try using the r3.2xlarge instance type. You lose AVX2, and 100 MHz of base clock, but you get the same memory for 38% of the price. It only has 8 virtual cores, but there's a good chance your hardware mates aren't running memory bandwidth intensive applications, so those 8 virtual cores can be fully fed.

For instance, we use r3.2xlarge for memcache. Our memory bandwidth usage is limited by the network bandwidth, and we keep spare capacity by not maxing out the network bandwidth. Our CPU usage is around 10%, and the memory bandwidth usage is less than 1 Gb/s, out of the 59.7 Gb/s the E5-2670 v2 chip has available.

[nucleon]

This is what I love about ec2.

I don't need to go out and buy hardware to test a theory. I just run up an instance :)

Ok, starting up an r3.2xlarge and going to experiment with that, and see how it compares.

[nucleon]

For those that don't know, you can bid a 'spot price' and if the market rate of a particular instance falls below your bid price - an instance of that type is allocated to you.

Doing some price analysis over last 7 days of the market price of the two instance types discussed above - c4.8xarge and r3.2xlarge I get the attached pic.

What I did, I grabbed the last 7-days, per-hour rates of different zones (rows 43-62, 66-85), I worked out the 50-percentile, 75-percentile, and 90-percentile. (yellow bars). The first block is r3.2xlarge, the second block is c4.8xlarge.

The blue bars, are the aggregate figure for 7 days given 50%/75%/90% uptime. So how much it will cost you for the week based on bidding for 50/75/90% uptime.

I was bidding on r3.4xlarge and found the price too volatile for the cheapskate that I am. :) This was another reason for the c4 instance - I wanted to see how stable the prices were. The c4 instances seem a lot less volatile than the r3.4xlarge prices I was chasing.

From the graph, it looks like us-west1 (a or b) is the better value for r3.4xlarge. I'm current in us-east1c zone with my c4 instances.

[Xyzzy]

http://www.businessinsider.com/amazo...nothing-2015-4