X79 chipset

[TheMawn]

Originally Posted by henryzz

Only because it is quad channel.
2133 is the slowest that JEDEC is planning to standardize for DDR4. It will actually be slower than 2133 DDR3.

I don't understand. DDR4 is not necessarily quad channel, just like DDR3 is not necessarily dual or triple or quad channel. That's dependent on the motherboard.

I did forget that 2133 MHz DDR3 is actually 1066 MHz base frequency doubled for DDR3, so there's some ambiguity as to what 2133 MHz DDR4 actually is. However, the bandwidth itself is still doubled from DDR3 to DDR4 just like it was from DDR to DDR2 and to DDR3. I don't understand how 2133 MHZ DDR4 can be slower than 2133 MHz DDR3.


EDIT: Oh wow. I didn't realize DDR4 was going to be single-channel only. I of course will wait and see but I don't have a good feeling about dropping the multiple DIMM's per channel.

[kracker]

Indeed. I don't have a good feeling about this if it is only single channel, unless there is something we don't know yet.

[kladner]

Originally Posted by kracker

Indeed. I don't have a good feeling about this if it is only single channel, unless there is something we don't know yet.

I had thought that a better description is "one DIMM per channel." However, aren't multiple channels still possible?

I vaguely remember being puzzled by this concern when I first read about DDR4.

[kracker]

Originally Posted by kladner

I had thought that a better description is "one DIMM per channel." However, aren't multiple channels still possible?

I vaguely remember being puzzled by this concern when I first read about DDR4.

To be honest, I'm not completely sure.
Well... off I go "researching".

[danaj]

Speculation, but this Extremetech article shows the Haswell-E having 4 channels (1 DIMM per channel). This seems to allow a bit more flexibility on the board side, but also maybe more expensive to have more slots. I bet there will be more 2-slot systems, to save money on the controller with the extra density still allowing high-total-memory setups.

Samsung's brochure and this old Anandtech article seem to indicate the approximate parity in speed for fast DDR3 and initial DDR4 offererings. There are a lot more gotchas I'm sure. It'll be interesting to see.

[TheMawn]

Well yeah. In any performance oriented system nobody will have the guts to sell DDR4 modules that are worse than DDR3. With even 2400 MHz becoming very affordable in the last year or so the minimum standard for DDR4 got pushed up since they want to be at least as good as an upper-standard DDR3.

Of course, that means double the bandwidth of a DDR3 module to even match dual-channel. I don't know what they're going to do as far as quad-channel is concerned. I guess it's still a matter of making a memory controller that can handle 8266 MT/s (2133 MHz x 4) and hoping the modules can handle it.

Manufacturers seem to be going on and on about how 1.05V - 1.20V is going to be a standard voltage for DDR4. I can see manufacturers setting up a decent heatsink and trying to get much more speed by upping the voltage to 1.3V - 1.5V if the chips can take it.


Still, I don't know if they're targetting single or double channel DDR3 as the baseline for DDR4. It seems like much more of a mobile market sort of thing.

[danaj]

Originally Posted by TheMawn

Well yeah. In any performance oriented system nobody will have the guts to sell DDR4 modules that are worse than DDR3. With even 2400 MHz becoming very affordable in the last year or so the minimum standard for DDR4 got pushed up since they want to be at least as good as an upper-standard DDR3.

Of course, that means double the bandwidth of a DDR3 module to even match dual-channel. I don't know what they're going to do as far as quad-channel is concerned. I guess it's still a matter of making a memory controller that can handle 8266 MT/s (2133 MHz x 4) and hoping the modules can handle it.

Take a DDR3-2400 quad channel solution like Intel 2011, 19.2 GB/s x 4 max. Now DDR4-2400 with 4 modules and like the Haswell-E documents, a controller with 4 point-to-point links. Single channel for each DIMM so each channel is the same 19.2 GB/s but there are 4 links, so 19.2 x 4 GB/s.

These are the same -- what am I missing? (this is ignoring a bundle of real life details, e.g. banks, data layout, latency, bus overheads, etc.).

Each module doesn't have to be double the bandwidth. There are multiple links each operating independently. High end servers might even have more links. It could be that the 4-link solutions will make it to low end systems, so we'd end up with an increase in overall bandwidth. Let me make an aside that switches that in theory should have Nx1Gb/s bandwidth usually have a small total throughput because of hardware/firmware limitations / cost-cutting. The same thing could happen with the memory controller (though Intel manages quad-channel today).

If we're comparing high end desktops (e.g. LGA2011), then it seems mostly like a wash at first. 4 channel DDR3 vs. 4-link DDR4. Lower power (dubious value for these desktops?). higher density (just offsetting the 8 slots -> 4 slots), faster modules in the future, higher cost.

[TheMawn]

Some of the terminology might be a bit confusing. We may be using the same word to mean different things, or different words to mean the same thing. Let's see if I can clear that up a bit.

In dual-channel, you can use two DIMMs as a pair and the data is spread across both. The memory controller simultaneously accesses both DIMMs and each provides "half" the data and thus bandwidth is doubled. In LGA 2011, this is quadrupled instead of doubled as a set is formed from four DIMMs instead of two.

A typical dual channel system does have four DIMM slots, but populating all four does not help because while the memory controller can access both DIMMs within a set at the same time, it cannot access both "sets" at the same time. In the same way, a quad-channel system CAN have eight DIMMs but the memory controller can only access one "set" of four DIMMs at a given time.

With DDR4 dropping the multiple DIMMs per channel thing, there is no such thing as dual-channel or triple-channel or quad-channel anymore, which means that while you have four DIMMs, each one is its own "set" and the memory controller cannot access more than one stick at a time.

For this reason, the DDR4 module must be twice as fast as a DDR3 module to break even, because dual-channel is in literally every modern system.


I've given your post some thought and if you've done more reading than I have, it's possible that I'm the one who is confused. The only way I can think of is that DDR4 drops the concept of multiple DIMMs per channel but introduces a system where each DIMM is accessed independently and simultaneously but that doesn't really make any sense either.

This is akin to putting your pictures on one drive, your videos on another, music on a third and applications on a final, instead of making a big RAID array and having all four of those things on the RAID array. There is a difference. In the RAID array, your access to your pictures is quadrupled if you're not also accessing your videos, music or applications. In the non-raid array, your four drives still exist as independent entities and CAN all be accessed at the same time, but only if you need a picture, a video, a song and a program all at the same time.


You only get to quadruple your bandwidth if you spread your data across all four DIMMs, and doing that is the very point of multi-channel memory. If DDR4 is dropping the multi-channel thing but is somehow continuing to spread data across DIMMs, then it is just giving the concept a new name.

[danaj]

Originally Posted by TheMawn

The only way I can think of is that DDR4 drops the concept of multiple DIMMs per channel but introduces a system where each DIMM is accessed independently and simultaneously but that doesn't really make any sense either.

This is my take on what is happening, so this is where our disconnect is. For instance:

Originally Posted by Wikipedia, a really weak reference

DDR4 also anticipates a change in topology. It discards the multiple DIMMs per channel approach in favor of a point-to-point topology where each channel in the memory controller is connected to a single DIMM.

The idea being that if we had a 4-channel MC before, we still have 4 channels but now each connects to a separate DIMM via a point-to-point bus. Based on the article that sources from (bit-tech), one of the goals was to push the parallelism almost entirely into the memory controller, simplifying the bus and DIMMs. It seems to follow on the parallel-bus -> multiple-serial-buses trend that so many things have been doing. That page also has a diagram of how a server might use switches to add more slots.

At least that's my take. I don't see why the memory controller, which has 4 independent connections to 4 independent DIMMs, would have to only access one at a time. If it did, performance would be horrible compared to current systems. The server folks are presumably willing to make some compromises for less power + more density + less cost (eventually), but taking a 4x bandwidth hit would be something everyone would be talking about.

[fivemack]

Originally Posted by TheMawn

With DDR4 dropping the multiple DIMMs per channel thing, there is no such thing as dual-channel or triple-channel or quad-channel anymore, which means that while you have four DIMMs, each one is its own "set" and the memory controller cannot access more than one stick at a time.

You've got that the wrong way round: in a DDR4 system there are four DIMMs, each one has its own controller, and the set of four memory controllers can access all four DIMMs simultaneously. There are no longer multiple DIMMs per channel, there are only multiple channels.

What you can't do with DDR4, and do routinely do with DDR3, is have two DIMMs on the same controller. The DDR controller can only access one of the DIMMs attached to it at a time, but in a four-channel configuration there are four controllers and either A or B, plus either C or D, plus either E or F, plus either G or H, can be accessed simultaneously.

[TheMawn]

Got it. Makes much more sense.