mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware

Reply
 
Thread Tools
Old 2020-12-02, 03:11   #144
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

7×701 Posts
Default

https://en.wikipedia.org/wiki/MCDRAM It's not just on the board, it's in the same package as the processor cores. Like modern caches. https://en.wikipedia.org/wiki/Xeon_P...QHL6_DSCx3.jpg
kriesel is online now   Reply With Quote
Old 2020-12-02, 03:38   #145
ewmayer
2ω=0
 
ewmayer's Avatar
 
Sep 2002
República de California

1160610 Posts
Default

Quote:
Originally Posted by kriesel View Post
https://en.wikipedia.org/wiki/MCDRAM It's not just on the board, it's in the same package as the processor cores. Like modern caches. https://en.wikipedia.org/wiki/Xeon_P...QHL6_DSCx3.jpg
Yes, I took 'onboard' to mean in the CPU package proper, because for it to be used to imply 'on the (mother)board' would be inane - as opposed to 'onshore'? - but apparently the mobo manual's usage means just that. AFAICT the MCDRAM frequency is not fiddlable, unless it's somehow tied to the CPU frequency. Anyhow, found this doc from Colfax Research - the folks who made the GIMPS KNL workstation model - re. the 3 usage modes (I've added numbering for clarity):
Quote:
Advantages and disadvantages of the three modes are:

[1] Cache mode — No work required to use, but may have lower performance than flat mode in case of frequent misses in HBM as cache.
[2] Flat mode — May offer better performance than cache mode, but requires modifications of the code and/or execution environment.
[3] Hybrid mode — Benefit of both Flat mode and Cache mode, but smaller sizes for each.
Since I have 0 RAM aside from the MCDRAM, HBM-as-cache misses are not in play, thus choosing any usage mode other than [1] would be silly: if 0 cache misses in mode [1], then [2] offers no performance benefit and requires both the prorietary Intel toolchain and major recode, and [3] offers the worst of both worlds: smaller L3 cache than 16GB, and needing recode.

Last fiddled with by ewmayer on 2020-12-02 at 03:40
ewmayer is offline   Reply With Quote
Old 2020-12-02, 06:01   #146
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

26·73 Posts
Default

I interpreted that Colfax document as explaining that cache mode would really be a cache, and not seen by the OS as main memory; while flat would treat the on-package RAM as main memory but a separate space from the DDR4 modules- it's the "two main memories" that I thought would require special coding.
In your case, with no DDR to code for, I expected flat to be the choice.

Interesting that "cache" mode is still seen by the OS as main memory.
VBCurtis is offline   Reply With Quote
Old 2020-12-02, 20:09   #147
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

132B16 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
I interpreted that Colfax document as explaining that cache mode would really be a cache, and not seen by the OS as main memory; while flat would treat the on-package RAM as main memory but a separate space from the DDR4 module.
I concur after a light reading of it. Somehow the BIOS, Windows 10 and prime95 knew what to do with the MCDRAM in the absence of any DIMMS in my system, and use it as addressable memory. Sounds like a 1-legged flat mode imposed by the OS to me. Windows and Linux and prime95 and presumably Mlucas have been dealing with multiple NUMA banks for a while now. Looking at both source codes for how dual processor package systems are handled might be educational.

Last fiddled with by kriesel on 2020-12-02 at 20:17
kriesel is online now   Reply With Quote
Old 2020-12-02, 20:33   #148
ewmayer
2ω=0
 
ewmayer's Avatar
 
Sep 2002
República de California

2·7·829 Posts
Default

Just did a reboot and had a look at the CPU Configuration menu in the BIOs:

o CPU freq = 1.4GHz and not BIOS-adjustable (nor the min.max ratios), L1 = 4352KB (4.25MB), L2 = 34816KB (34MB),

o HT is enabled by default, since I found no gain from using it in my Mlucas timing tests, disabled it to see if that makes a difference for my current 1-thread-per-core running

Under the Advanced Power Management Configuration Power Technology menu:

o Config TDP submenus not shown

o EIST (P-states): "EIST (Enhanced Intel SpeedStep Technology) allows the system to automatically adjust processor voltage and core frequency to reduce power consumption and heat dissipation. The options are Disable and *Enable*" -- basically, a form of auto-throttling, I disabled, which should be safe to do with my case in side-panel-removed form with the resulting low CPU temps (I ordered the baking sheet as my first side-opening mesh-covering solution attempt, btw). Note the sidebar infopane adds "When disabled, CPU frequency is set at max non-turbo", which means the aforementioned 1.4GHz. But "non-Turbo" implies there might be an available "turbo" mode - that might refer to a particular version of the CPU which has the clock unlocked, though.

o CPU C State Control -- see nothing of interest there

o CPU T State Control (Available when Power Technology is set to Custom): has a single submenu, ACPI (Advanced Configuration Power Interface) T-States: "Select Enable to support CPU throttling by the operating system to reduce power consumption. The options are *Enable* and Disable" -- switched to Disable, saved changes, booted the OS, fired up big 64M-FFT F30 job on 64 cores, wait for next ~10-min checkpoint update ... and no discernible speed change. So unless someone else has any bright ideas, looks like we are more or less maxed out already at the default settings.
ewmayer is offline   Reply With Quote
Old 2020-12-02, 20:40   #149
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

7·701 Posts
Default

I did not mess with the BIOS in regard to cpu speed limits, that I recall. Windows 10 Task Manager has shown it at as I recall at least 1.42, 1.44, and 1.5Ghz; the first two under heavy prime95 load and side cover off. So Ernst you may want to look to CentOS or reconsider SpeedStep. Reviewing the thread, I find
1.05 https://mersenneforum.org/attachment...9&d=1605147324
1.13 https://mersenneforum.org/attachment...3&d=1605231308
1.4 https://mersenneforum.org/attachment...1&d=1605151520
1.42 and 1.44 observed https://mersenneforum.org/showpost.p...&postcount=124
1.5 https://mersenneforum.org/attachment...2&d=1606566337

Disabling may be a plus for stable clock during benchmarking. But subtracting a bit of throttle-UP at times.

Re the baking sheet, that may keep the cats out, but as silicone coated fiberglass will do very little for EMI control.

Last fiddled with by kriesel on 2020-12-02 at 20:57
kriesel is online now   Reply With Quote
Old 2020-12-05, 16:44   #150
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

7·701 Posts
Default

Highest indicated clock rate I've seen from the Xeon Phi 7250 is 1.55 Ghz (Windows Task Manager, a small fraction of the cores in use). Square waves have odd harmonics of the fundamental frequency with declining amplitude. So up to 1.55, 4.65, 7.75....Ghz at amplitude 1, 1/3, 1/5,... How high should be blocked by screening to replace the side panel?
The removable side panels are nonmagnetic, as are top, front, and bottom. Back panel and power supply case are magnetic; back panel has about 200 quantity 6mm hexagonal holes closely spaced for ventilation. Presumably that constitutes a sufficiently bad slot antenna. Assume that perforated aluminum or steel for a new side panel can have the same size hole (that electrical vs. magnetic shielding is not an issue).

Slots are of order 0.1 wavelengths wide by 0.5 wavelengths long in slot antenna use.
6mm/0.1 = 60mm corresponding to an estimated upper bound of 300e6m/sec/.06m = 5Ghz, only 3.2 times the maximum observed clock rate, which seems rather low.

Typical microwave oven frequency is 2.45GHz. There the shielding has holes ~2mm in size. 0.002m / ( 300e6m/s / 2.45e9/s ) = .002/.122 = 0.016 wavelengths.

These results suggest that for somewhat quieting the EMI from a miner frame system, with 3+Ghz CPU and up to ~1.8Ghz gpus, hardware cloth with its 1/4" wire pitch would be a much better choice than 1"x2" aperture welded wire fence roll, for constructing a 5-sided cover shield.

Last fiddled with by kriesel on 2020-12-05 at 16:58
kriesel is online now   Reply With Quote
Old 2021-02-03, 07:33   #151
octonion
 
Feb 2021

2·3 Posts
Default

I just bought one of these Xeon Phi workstations - does anyone know if the Supermicro K1SPE motherboard might be capable of supporting 128GB 8Rx4 (rank 8) LRDIMMs? It's listed as supporting 64GB LRDIMMs, for a total of 384GB of RAM, but I'm wondering if it might be capable of 768GB of RAM.
octonion is offline   Reply With Quote
Old 2021-02-04, 23:33   #152
ewmayer
2ω=0
 
ewmayer's Avatar
 
Sep 2002
República de California

101101010101102 Posts
Default

Quote:
Originally Posted by octonion View Post
I just bought one of these Xeon Phi workstations - does anyone know if the Supermicro K1SPE motherboard might be capable of supporting 128GB 8Rx4 (rank 8) LRDIMMs? It's listed as supporting 64GB LRDIMMs, for a total of 384GB of RAM, but I'm wondering if it might be capable of 768GB of RAM.
Search for "contact" in the Mobo manual PDF, that shows contact info on p17, Tech Support e-mail is support@supermicro.com .

You don't happen to have one of those monster 128GB modules lying around which you could plug in and see-if-it-boots-and-recognizes-the-RAM, do you?

Just curious, roughly what $ are we talking about for 768GB of such RAM?
ewmayer is offline   Reply With Quote
Old 2021-02-06, 00:43   #153
octonion
 
Feb 2021

2×3 Posts
Default

I'm trying to find someone with a 128GB DDR4 8Rx4 LRDIMM module to borrow for about a week.
octonion is offline   Reply With Quote
Old 2021-02-11, 05:49   #154
octonion
 
Feb 2021

1102 Posts
Default

Sadly, I can't get my Xeon Phi workstation to display output. Reseated the power connections, checked the jumpers, cleared the CMOS and reset the BIOS, tried three different monitors with VGA input. It may simply be a bad VGA cable, despite it being brand new. I'd hate to have to return it.
octonion is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
AMD vs Intel dtripp Software 3 2013-02-19 20:20
Intel NUC nucleon Hardware 2 2012-05-10 23:53
Intel RNG API? R.D. Silverman Programming 19 2011-09-17 01:43
AMD or Intel mack Information & Answers 7 2009-09-13 01:48
Intel Mac? penguain NFSNET Discussion 0 2006-06-12 01:31

All times are UTC. The time now is 03:54.

Sun Feb 28 03:54:53 UTC 2021 up 87 days, 6 mins, 0 users, load averages: 1.39, 1.56, 1.60

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.