mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Hardware (https://www.mersenneforum.org/forumdisplay.php?f=9)
-   -   Intel Xeon PHI? (https://www.mersenneforum.org/showthread.php?t=25767)

Cheetahgod 2020-07-23 05:10

Intel Xeon PHI?
 
It would be interesting to see if you could use a intel phi co-processor to do prp tests.
You can get them on ebay for around $80 - $100

[url]https://www.ebay.com/itm/Intel-Xeon-Phi-71S1P-8GB-RAM-1-1GHz-61-core-CoProcessor-PCI-e-similar-to-7110P/223708534309?hash=item3416115625:g:MUwAAOSwHSxdp2xv[/url]

Uncwilly 2020-07-23 14:08

It is a Xeon.
You might want to look at this thread: [url]https://www.mersenneforum.org/showthread.php?t=16912[/url]

kriesel 2020-07-23 14:56

[QUOTE=Cheetahgod;551340]
[URL]https://www.ebay.com/itm/Intel-Xeon-Phi-71S1P-8GB-RAM-1-1GHz-61-core-CoProcessor-PCI-e-similar-to-7110P/223708534309?hash=item3416115625:g:MUwAAOSwHSxdp2xv[/URL][/QUOTE]Any model that ends in P is passively cooled, expecting the case it's installed in to blow cooling air through it. Note there's no fan visible in the listing you posted. The models ending in A that have their own cooling fan suitable for consumer chassis use are rare but do exist.
There are multiple generations of Xeon Phi with substantially different performance. [URL]https://en.wikipedia.org/wiki/Xeon_Phi[/URL]

Evaluating the 7120A I bought is on my to-do list. [URL]https://www.ebay.com/itm/Intel-Xeon-Phi-7120A-PCI-Express-61-CORE-Coprocessor/233581462183[/URL]

Other listings for the same model with stock on eBay are currently at US$600 and up.
[URL]https://www.ebay.com/itm/New-Dell-Intel-Xeon-Phi-Coprocessor-3120A-6GB-1-1GHz-57-CORES-240GB-s-NY25D/192724974858[/URL] is a less costly possibility, with its own fan.

Otherwise figure out how you're going to cool a passively cooled model.

Systems designed to support the passively cooled Xeon Phis can be expensive [URL]https://www.ebay.com/itm/SuperMicro-4U-Server-7048GR-TR-2x-E5-2680v3-Xeon-Phi-7120P-256GB-WX-690LP1-MORE/254659755925[/URL]

To compile code for Xeon Phi and their embedded Linux OS requires special development software from Intel which is $3000/license, or a busy 30-day free evaluation period.

I estimate it to be somewhere around 1/4 the speed of a Radeon VII, yet use an equal power budget.

tServo 2020-07-23 20:58

GIMPS had a Xeon Phi system ( at one time )
 
A bunch of people chipped in and a system was purchased.
As mentioned, the software license has probably expired.
The person who hosted the system is inactive in this forum so I'm not sure the system can be accessed.
Ernst did a lot of work on the system, as I recall. However, the results from this hardware were unimpressive.

See this thread:

[URL="https://www.mersenneforum.org/showthread.php?t=21544"]https://www.mersenneforum.org/showthread.php?t=21544[/URL]

BTW, I have seen blog posts showing IBM is getting ready to try again with a massive new chip. Perhaps third time's a charm.
I wonder who would bother to try it since their reputation is somewhat tarnished.

ewmayer 2020-07-23 22:04

[QUOTE=tServo;551380]A bunch of people chipped in and a system was purchased.
As mentioned, the software license has probably expired.
The person who hosted the system is inactive in this forum so I'm not sure the system can be accessed.
Ernst did a lot of work on the system, as I recall. However, the results from this hardware were unimpressive.[/QUOTE]

The KNL was actually quite good in terms of performance for the work I was using it for - one of my two side-by-side (one @60M FFT, one @64M) Pépin primality tests of F30, the 64M was running 64-threaded (i.e. 1 thread per hardware core) on the KNL, getting ~68 ms/iter at time the physical host of the system, David Stanfill, went AWOL. Getting good || scaling for this kind of GIMPS-style work on more than 8-16 cores was a problem, but I found that the bigger the FFT memory-footprint, the better the || scaling, so at 64M I was able to get (IIRC) more than 50% the extrapolated "perfect parallel" scaling. I expect the above runtime has since been eclipsed by top-tier GPUs, but not bad for a $5000 system at the time we bought it. And even more valuable was the ability to do AVX-512 code prototyping & testing on actual hardware, rather than a simulator.

Re. licenses, I only ever used GCC on the system, what proprietary licenses might be needed in that context? Would a standard Linux install work on this kind of hardware?

VBCurtis 2020-07-23 22:09

If I understand Phi correctly, the chip linked in the first post is on a PCI card and needs a special compiler and/or OS, while the system GIMPS purchased was the sole CPU on a motherboard and ran a "regular" linux and GCC.

The cheap used parts are the addin cards, and are much tougher to get performance out of; I would be surprised if a Phi outran a same-price CUDA card on any software run for projects in this forum. Cool tech from 5+ years ago may still be cool now, but is almost-never a faster solution than new commodity stuff.

kriesel 2020-07-24 02:40

[QUOTE=ewmayer;551384]Re. licenses, I only ever used GCC on the system, what proprietary licenses might be needed in that context? Would a standard Linux install work on this kind of hardware?[/QUOTE]For the addin PCIe Xeon Phi, there's a uOS linux variant embedded on the card. Drivers to talk to it from the host OS are needed Or something like that. Supposedly there are Windows drivers for it. There appears to also be Linux host support. The license is for Intel's Parallel Studio and related tools.

Puget Systems has some old articles about it. [URL]https://www.pugetsystems.com/all_hpc.php?query=phi[/URL]

ewmayer 2020-07-24 20:58

[QUOTE=kriesel;551398]For the addin PCIe Xeon Phi, there's a uOS linux variant embedded on the card. Drivers to talk to it from the host OS are needed Or something like that. Supposedly there are Windows drivers for it. There appears to also be Linux host support. The license is for Intel's Parallel Studio and related tools.

Puget Systems has some old articles about it. [URL]https://www.pugetsystems.com/all_hpc.php?query=phi[/URL][/QUOTE]

Well, for under $100, it seems worth a try - I'd snap it up but I'm fresh out of full-width PCI slots in my 2 systems.

I just contacted the seller re. driver and other-software support, reply said they'd been referring people to [url=software.intel.com/content/www/us/en/develop/articles/intel-manycore-platform-software-stack-mpss.html]this Intel page[/url] -- Looks like both Linux and Windows support stacks there, The Linux ones involve specific enterprise-server distros, but I see no licensing hurdles.

chalsall 2020-07-24 21:05

[QUOTE=ewmayer;551482]The Linux ones involve specific enterprise-server distros, but I see no licensing hurdles.[/QUOTE]

One could reasonably presume that includes RedHat, which is (legally) cloned in CentOS.

You only really need to pay for it when the PHB's insist on it (doesn't hurt that they're now owned by IBM).

Xyzzy 2020-07-25 12:52

[url]https://developers.redhat.com/blog/2016/03/31/no-cost-rhel-developer-subscription-now-available/[/url]

ewmayer 2020-07-25 21:36

Mike, this sounds like the kind of software-stack-to-run-interesting-hardware project you enjoy, especially with the GPUs you tried out last month idled due to excess heat. If someone manages to get one of these up and running, and the throughput from running mprime or Mlucas on all cores is within 10x of a Radeon VII - which seems entirely plausible, based on my earlier runs on the GIMPS KNL, you've already broken even in terms of what even a used R7 currently costs, assuming the wattage is similar. C'mon, you know you want to...

ewmayer 2020-07-26 00:00

Mike just reminded me that Phi copros are passively cooled, so one would need some serious external-fan-air to cool them. Or, [url=https://www.allied-control.com/xeon-phi-immersion-cooling-concept/]one could go high-tech[/url]. :)

kruoli 2020-07-26 12:38

I bought a 31S1P four and a half years ago (52,21 €, including shipping and VAT, new). I do not know why it was so extremely cheap at that time. I'll might be able to get two more of them from former fellow students, which also bought some.

The Solution to the fan problem was a 3D printed adapter for fans on one end. The ones I used originally were to slow so upgraded them later on. The 3D model was taken from thingify without edits. A video of my build is visible on YouTube:
[YOUTUBE]9qpGcjhf7dM[/YOUTUBE]

VBCurtis 2020-07-26 14:58

[QUOTE=kruoli;551641]I bought a 31S1P four and a half years ago (52,21 €, including shipping and VAT, new). [/QUOTE]

What have you run on it?

kruoli 2020-07-26 15:26

For my student job at the university I was given the task to vectorize a given program that simulated diffraction. It was required to run best on CPUs and should run using OpenCL. I had some of that running on it, also, but without OpenCL. The only problem was that each thread required quite large amounts of memory when you were simulating longer time durations, so it was not well suited for that card.

Later on I implemented a generator for [URL="http://mathcs.clarku.edu/~djoyce/mpst/surprising/"]surprising strings[/URL] and tried multiple things to optimize my code on that. After two months or so I dismounted the machine and never used the card again, since.

Aside from minor things I never used the card nearly to its potential. So when I saw this thread, I liked the idea of reviving the card.

ewmayer 2020-07-26 22:01

1 Attachment(s)
[QUOTE=kruoli;551641]The Solution to the fan problem was a 3D printed adapter for fans on one end. The ones I used originally were to slow so upgraded them later on. The 3D model was taken from thingify without edits.[/QUOTE]

For my recent 3 x Radeon7 open test-frame build, once I housed the completed beast at back of my desk under an open-sided glass-top monitor stand (which I jury-rigged to double the original 4" max adjustable-leg height, even with free airflow from the front and sides the R7s were throttling big-time (and they were downclocked to the Linux/ROCm sclk = 3 setting, which translates to ~150W per card). My solution was to buy a compact corded (= serious airflow, none of this wimpy-ass USB-fan crap) desktop fan which I placed at left edge of monitor stand to blast air across the machine - see pic below. That gave good enough airflow that the GPU at the left end, the one closest to fan whose triple-air-intake gets air directly from the desktop fan, doesn't even need its own fans - even at the higher sclk=4 (200W) setting, its fan are basically idling. One could do a similar build with a Xeon Phi card in that location, except I don't know if the Phi's airflow needs would be met with the same kind of positioning - based on the PCI-bus location is eBay seller pics, replacing the leftmost R7 in my pic with a Phi would put the blue decorative top plate facing the fan - is that designed to dissipate processor heat or not? How about air hitting the circuit-board-revealing bottom face of the Phi at a right angle?

kriesel 2020-07-27 00:10

I have a large-room air purifier (HEPA filter) blowing its exhaust at my Radeons.

kruoli 2020-07-27 08:23

Unfortunately, the 31S1P is only open on the two smallest sides. Even my setup with the two 40mm fans (found the [URL="http://www.thingiverse.com/thing:997213"]link[/URL] again) was not enough for continuous full blast calculations. I have not tried to open up the metal housing, maybe one could cool it better that way. Now, since I'm out of warranty, I'll might try that.

kriesel 2020-07-27 15:42

[QUOTE=kruoli;551718]Unfortunately, the 31S1P is only open on the two smallest sides. Even my setup with the two 40mm fans (found the [URL="http://www.thingiverse.com/thing:997213"]link[/URL] again) was not enough for continuous full blast calculations. I have not tried to open up the metal housing, maybe one could cool it better that way. Now, since I'm out of warranty, I'll might try that.[/QUOTE]Either that or shop around for some high static pressure blowers.

ewmayer 2020-07-27 22:23

[QUOTE=kruoli;551718]Unfortunately, the 31S1P is only open on the two smallest sides. Even my setup with the two 40mm fans (found the [URL="http://www.thingiverse.com/thing:997213"]link[/URL] again) was not enough for continuous full blast calculations. I have not tried to open up the metal housing, maybe one could cool it better that way. Now, since I'm out of warranty, I'll might try that.[/QUOTE]

Re. open the housing - That was my next question. :) Looking at the eBay seller's pic, looks like you simply need to remove a couple easily-accessible screws at each corner to pull the blue-plate-special part of the housing off.

LaurV 2020-07-28 02:21

[QUOTE=kruoli;551641]I bought a 31S1P ...[/QUOTE]
We fixed your youtube tag, see the reason in the post.

kruoli 2020-07-28 08:19

[QUOTE=ewmayer;551748]Re. open the housing - That was my next question. :)[/QUOTE]

I'm going forward to do this this week and report on it here.

[QUOTE=LaurV;551763]We fixed your youtube tag, see the reason in the post.[/QUOTE]

Oops... Thank you! :smile:

kruoli 2020-08-03 17:13

2 Attachment(s)
Here are my results of the disassebly: Only two screws to loosen, but the remaining parts cannot be "blown on". The wind tunnel is closed. In order to open it, I'd have to bend the fins and I'm not comfortable doing this.

masser 2020-08-03 18:19

How large is the area on top? Could you secure a large (200 mm) fan on top that would draw air through the gaps in that heatsink?

kruoli 2020-08-03 18:45

92 [$]\cdot[/$] 246 mm (total area).
76 [$]\cdot[/$] 220 mm (only the fins).

My only 200 mm fan is quite slow and would overlap my Phi by a bunch. Maybe two or three 80 mm? But I doubt that any fan could suck enough air through those very tightly fitted fin tops.

ewmayer 2020-08-03 21:02

Thanks for the pics, now I finally have a good sense what we're dealing with, colling-options-wise. As a short-term nondestructive half-measure, setting it crosswise in the airstream from a corded desktop fan [url=https://www.amazon.com/gp/product/B001R1RXUG/]like I use to help cool my 3-Radeon-VII desktop cruncher[/url] couldn't hurt, just see how rapidly temps spike under load and how bad the resulting throttling is.

But what this beast really needs is a larger version of the kinds of centrifugal-ducted fan in a U-shaped enclosure one finds in laptop and other compact compute devices. Something [url=https://www.amazon.com/Soplador-radial-centrigual-sin-escobillas/dp/B01CSNEO2G]like this[/url], just of the closest size you can find to match the dimensions of the rectangular opening of the finned end of your card, to minimize or eliminate the need for any custom adapter-ducting segment.

kriesel 2020-08-04 00:03

7120A uncovered
 
3 Attachment(s)
Fan housing is cast metal in this unit.

ewmayer 2020-10-27 19:54

Quick update - the wares on offer by the eBay seller linked in the OP are all gone, but the specific model of Phi in question was an older [url=https://en.wikipedia.org/wiki/Knights_Landing_microarchitecture]Knights Corner one[/url], which supports a flavor of 512-bit SIMD, but not AVX-512. For that you need a Knights Landing, which means a 72**-model Xeon Phi, and those tend to be both harder to find in the aftermarket and pricier, [url=https://www.ebay.com.au/itm/Intel-SR2X5-Xeon-Phi-Processor-7210F-64-core-32MB-cache-nice-/153512705068]even in copro form[/url].

If someone spots one for under US$500, tho, by all means send up a flare. Could be an interesting mini-build project for next year.

EdH 2020-10-27 19:59

[QUOTE=ewmayer;561277]. . .
If someone spots one for under US$500, tho, by all means send up a flare. Could be an interesting mini-build project for next year.[/QUOTE]
Not knowing anything about this topic, is this one?:

[URL]https://www.ebay.com.au/itm/Intel-Xeon-Phi-Processor-7210F-1-30Ghz-64-Core-SR2X5-867304-001/293687842505[/URL]

ewmayer 2020-10-27 20:15

[QUOTE=EdH;561278]Not knowing anything about this topic, is this one?:

[URL]https://www.ebay.com.au/itm/Intel-Xeon-Phi-Processor-7210F-1-30Ghz-64-Core-SR2X5-867304-001/293687842505[/URL][/QUOTE]

[I snipped all the link-tracking crud from your URL] That's just the bare CPU ... and I just realized that my "pricier" link is also bare-CPU-4-sale. So your bare-CPU is a far better deal than the one I linked to, but in any event we need the pci-e coprocessor version, which looks like the one linked in the OP, just with a newer KNL-version CPU.

EdH 2020-10-27 20:25

[QUOTE=ewmayer;561280][I snipped all the link-tracking crud from your URL] That's just the bare CPU ... and I just realized that my "pricier" link is also bare-CPU-4-sale. So your bare-CPU is a far better deal than the one I linked to, but in any event we need the pci-e coprocessor version, which looks like the one linked in the OP, just with a newer KNL-version CPU.[/QUOTE]
Thanks for the info and the help with trimming the URL. I did try to figure it out, but couldn't. I'm going to look at the two and maybe I'll understand the "URL extra crap (crud) better."

Edit: OK, I didn't even recognize "trkparams.":redface:
I will try to be more attentive. (Original URL trimmed.)

VBCurtis 2020-10-27 21:33

I've noticed these bare CPUs on ebay on occasion, but have never found a matching motherboard for a reasonable price, or even clearly and obviously compatible.
What boards work? I mean actually for sale, I've found a model listing but not a for-sale offer of the models I've seen that claim Phi compatibility.

On the used market, why would there be a supply imbalance between used CPUs for sale but not their corresponding boards (fried motherboards are a possibility, but seem unlikely for kit in this category)?

M344587487 2020-10-28 10:44

The standard 2nd hand server market has a lot of CPUs because companies often start with a low end configuration for a socket and upgrade to a high end configuration down the line as a cheap upgrade path. At least that's what I attribute it to.

kriesel 2020-10-28 13:38

FLARE!!
 
[QUOTE=ewmayer;561277]If someone spots one for under US$500, tho, by all means send up a flare. Could be an interesting mini-build project for next year.[/QUOTE]$499 before shipping, taxes, addins. Enclosure, modest PSU, 7210 64-core Xeon Phi with on-chip memory only, US power cord. No HD or OS included. Max add-in memory 6x64G LRDIMM or 6x32G RDIMM. 2x at max speed and density currently costs more than the base system. That will likely change over time. Max 6x64 +16 =400 GB, shy of the 512 GB suggested for F33 P-1, but ample for mersenne.org range.

[URL]https://www.ebay.com/itm/Hydra-7210-Workstation-Intel-Xeon-Phi-7210-CPU-Barebones-See-Desc/224194633867[/URL]
Get em while they last, 2 gone in past hour, 5 remain. (PM me where to send the finder's fee:])
Motherboard specs [URL]https://www.supermicro.com/products/motherboard/Xeon_Phi/K1SPE.cfm[/URL]

4 Phi 2U alternative $1950 [URL]https://www.ebay.com/itm/ASRock-Rack-Xeon-Phi-7210-Server-2U4N-F-X200/184460467766[/URL]

Tesla alternative for deep pockets & cold winters, just $7999. [URL]https://www.ebay.com/itm/Supermicro-SuperServer-1U-1029GQ-TXRT-w-4x-NVIDIA-Pascal-P100-GPU-FTY-Installed/373256448498[/URL]

xilman 2020-10-28 18:06

[QUOTE=ewmayer;561280] ... [/QUOTE]He's back!

VBCurtis 2020-10-29 23:16

I'm very interested to hear how the KNL system runs. Thinking about buying one for NFS work- sieving or matrices. Please let me know if you're interested in running a test or two so I can justify such a purchase for speeed!

ewmayer 2020-10-29 23:43

[QUOTE=VBCurtis;561496]I'm very interested to hear how the KNL system runs. Thinking about buying one for NFS work- sieving or matrices. Please let me know if you're interested in running a test or two so I can justify such a purchase for speeed![/QUOTE]

I, too, am very interested - actually I have a good idea how fast it should be for my Fermat-work, having hogged many system-days of CPU time on the GIMPS KNL before it went offline and its physical-host went AWOL. Main immediate issue for me will be how tricky the OS/software setup will be - gonna do the simple dumb thing first of attempting install of the same Ubuntu 19.10 image I've used for both my Radeon-VII-hosting systems as well as my 3 little Intel NUCs - if that happens to work in terms of being able to get a GCC Mlucas build and run-on-all-64-cores, we can get straight to testing stuff, happy to try out NFS for you if build&test are straightforward. Note, though, will be sticking to the 16GB of onboard RAM for the moment, so Block Lanczos on huge matrices is out. :)

Speaking of getting-up-and-running, just got e-mail from the KNL seller confirming it lacks onboard WiFi, so time to order a USB WiFi stick and an SSD.

VBCurtis 2020-10-30 01:43

I don't recall if msieve benefits from avx-512; perhaps a pair of compilation attempts, with vbits=256 and 512. Data set would be the one from the msieve benchmark thread- that matrix should be big enough to show Phi scaling up to 32 threads, and I can extrapolate from that test to larger matrices well enough.

The other test I have in mind is to run CADO on RSA-130 with 64 and 128 threads. Either one would run on I=13 (aka 13e equivalent on CADO), so memory use should be small enough for 32 or maybe 64 two-threaded processes to fit in 16GB. If performance scales with raw Ghz like it should for sieving, RSA-130 will take about an hour!

Strongly considering just buying this, and adding RAM later. There's some fun to be had on 16GB, with 6x16GB ram costing about as much as the initial CPU/board/case.

I think I'll wait for you to get ubuntu installed, and maybe wait for a benchmark. Hopefully I'll wait....

kriesel 2020-10-30 02:12

[QUOTE]actually I have a good idea how fast it should be for my Fermat-work, having hogged many system-days of CPU time on the GIMPS KN.[/QUOTE] Was that a 7210 also? Within 72xx, there's only modest clock rate differences, and core-count differences. 1.3, 1.4, 1.5 Ghz base, 64, 68, 72 cores. All 4-way HT. The bargain box is 1.3, 64, 4xHT. 1.5x72 is only ~1.3 x 1.3x64, not a big jump.

frmky 2020-10-30 04:23

[QUOTE=VBCurtis;561496]I'm very interested to hear how the KNL system runs. Thinking about buying one for NFS work- sieving or matrices. Please let me know if you're interested in running a test or two so I can justify such a purchase for speeed![/QUOTE]

Hijacking the thread, KNL works well enough for msieve LA if you are careful. Be sure to put one MPI thread on each tile. Sharing threads across tiles REALLY slows it down. And keep everything in MCDRAM if possible. Going out to DDR4 really slows it down. Of course there's only 16 GB of MCDRAM so that may not be possible with only one node. A few NFS@Home jobs have run on KNL, 2,1063+ being one example that used 8 nodes: [url]https://pastebin.com/Mdj1cyB2[/url]

tServo 2020-10-31 01:51

Speaking of the KNL systems, has anyone made a concerted effort to track down David Airsquirrels ? That system was purchased with donations from several forum members ( including me ) and should be retrieved to be used by someone.

Mark Rose 2020-10-31 03:07

AirSquirrels has been updating their Facebook and Instagram, so someone more official than me could inquire with them perhaps.

Mark Rose 2020-11-03 02:03

Support in Linux was just dropped. Make some noise on the mailing list if you want support restored in future kernel.

[url]https://lore.kernel.org/lkml/20201101131153.GA31235@kroah.com/T/#u[/url]

ewmayer 2020-11-03 19:30

Mod note: I moved/copy/edited the various KNL posts & material which had been posted to the neighboring XFX-4-sale thread to this one.

[QUOTE=Mark Rose;562019]Support in Linux was just dropped. Make some noise on the mailing list if you want support restored in future kernel.

[url]https://lore.kernel.org/lkml/20201101131153.GA31235@kroah.com/T/#u[/url][/QUOTE]

So KNL Linux-kernel support will be gone in the various Linux distros starting when? Immediately? Actually, though I don't really speak Linux-ese at the insier level, the relevant snip from that post seems to be:
[quote]The "big" driver removal is the MIC drivers have been asked to be
removed as the hardware never shipped and Intel no longer wants to
maintain something that no one can use. This is welcomed by many as the
DMA usage of these drivers was "interesting" and the security people
were starting to question some issues that were starting to be found in
the codebase.[/quote]
That refers specifically to hardware which never shipped, which does not apply to KNL. Can you clarify?

Should be moot for me in any event - as noted, I was planning on using a year-old Ubuntu distro for my initial boot attempt - should that fail for some reason there is a more-specific stack recommended by Intel for KNL, have the link saved somewhere.

bsquared 2020-11-03 19:39

[QUOTE=ewmayer;562118]Mod note: I moved/copy/edited the various KNL posts & material which had been posted to the neighboring XFX-4-sale thread to this one.



So KNL Linux-kernel support will be gone in the various Linux distros starting when? Immediately?

As noted, I was planning on using a year-old Ubuntu distro for my initial boot attempt - should that fail for some reason there is a more-specific stack recommended by Intel for KNL, have the link saved somewhere.[/QUOTE]

That discussion thread says MIC driver support is being removed. That's KNC, I believe, not KNL.

kriesel 2020-11-03 20:20

Knights Corner shipped (and I have one) MIC
Knights Landing shipped (and Dave Stanfil has one Ernst used) 2nd generation MIC
Knights Hill canceled, no indication it shipped; intended as 3rd generation MIC
Knights Mill launched 2017 [URL]https://ark.intel.com/content/www/us/en/ark/products/128694/intel-xeon-phi-processor-7235-16gb-1-3-ghz-64-core.html[/URL]
[URL]https://ark.intel.com/content/www/us/en/ark/products/128694/intel-xeon-phi-processor-7235-16gb-1-3-ghz-64-core.htm[/URL]

M344587487 2020-11-03 21:38

[QUOTE=ewmayer;562118]...
So KNL Linux-kernel support will be gone in the various Linux distros starting when? Immediately?
...[/QUOTE]Whatever they've removed they've removed from kernels >=5.10, 5.10 isn't released for another month and distros follow kernel releases in their own way. Most kernels are only supported for a few months except LTS kernels, the newest is 5.4 with support until 2025. LTS kernels get backports of mainly security updates, but even if a security flaw is present in what's been removed I believe they wouldn't backport a removal as it would potentially break userspace.

tl;dr you mentioned Ubuntu so if you need the thing use Ubuntu 20.04, it uses kernel 5.4 and it is itself an LTS release so is supported for years.

ewmayer 2020-11-11 20:36

The Hydra workstation KenK alerted me to just arrived from the eBay seller yesterday. It has 16GB onboard mem, no DIMMs at present, as 16GB is enough for my near-term needs. Installed a 1TB SSD and plugged in a Ubuntu 19.10 boot stick and a new WiFi USB, fired up, after Supermicro BIOS did its thing, <f11> started the Ubuntu boot, that quickly hit this:
[i]
(initramfs) Unable to find a medium containing a live file system
[/i]
'help' lists a bunch of available options. What should I try?

paulunderwood 2020-11-11 20:54

[QUOTE=ewmayer;562930]The Hydra workstation KenK alerted me to just arrived from the eBay seller yesterday. It has 16GB onboard mem, no DIMMs at present, as 16GB is enough for my near-term needs. Installed a 1TB SSD and plugged in a Ubuntu 19.10 boot stick and a new WiFi USB, fired up, after Supermicro BIOS did its thing, <f11> started the Ubuntu boot, that quickly hit this:
[i]
(initramfs) Unable to find a medium containing a live file system
[/i]
'help' lists a bunch of available options. What should I try?[/QUOTE]

On ddg'ing, one trick seems to use a USB2 slot rather than USB3.

[URL="https://askubuntu.com/questions/451662/when-i-start-the-ubuntu-iso-initramfs-says-unable-to-find-a-medium-containing"]Link[/URL]

ewmayer 2020-11-11 21:24

[QUOTE=paulunderwood;562931]On ddg'ing, one trick seems to use a USB2 slot rather than USB3.

[URL="https://askubuntu.com/questions/451662/when-i-start-the-ubuntu-iso-initramfs-says-unable-to-find-a-medium-containing"]Link[/URL][/QUOTE]

System has 5 USB ports - 2 in the case near the power switch, 3 in the ports area of the mobo backplate. Is there a simple way to tell the difference between USB2 and USB3, or must one wire-trace to where those connect to the mobo?

Wait - just did a websearch for the Supermicro K1SPE motherboard user manual, the Supermicro site was uncooperative re. a DLable pdf, but [url=https://www.manualslib.com/manual/1406892/Supermicro-5038k-I.html?page=11#manual]found this[/url]:
[i]
Four (4) USB 3.0/2.0 ports on the rear I/O panel (USB 2.0 0/1, USB 2.0 5/6 or
USB 3.0 2/3, USB 3.0 7/8)

One (1) Onboard USB header with two USB 3.0 connections (USB 3.0 8/9) for
front control panel[/i]

I only see 3 USB ports on rear I/O panel of my mobo, but sounds similar - unplugged the wifi stick from one of those - won't be needed until post-OS-install - to make room for the boot USB, rebooted ... and we have an install screen. Thanks! Will update once install (hopefully) completes.

paulunderwood 2020-11-11 21:30

[QUOTE=ewmayer;562932]System has 5 USB ports - 2 in the case near the power switch, 3 in the ports area of the mobo backplate. Is there a simple way to tell the difference between USB2 and USB3, or must one wire-trace to where those connect to the mobo?
[/QUOTE]

[URL="https://www.geeksinphoenix.com/blog/post/2014/01/12/How-to-tell-what-type-of-USB-connector-you-have-by-color.aspx"]USB 2.0 sockets are usually black and USB 3.0 blue.[/URL]

kriesel 2020-11-11 22:15

It's a different style case, but the following has a lot of info on K1SPE MB and BIOS
[URL]https://www.supermicro.com/manuals/superserver/mid-tower/MNL-1891.pdf[/URL]
(and yes it was well hidden)

downloads
[URL]https://www.supermicro.com/support/resources/resource_links.cfm[/URL]

ewmayer 2020-11-11 22:59

[QUOTE=paulunderwood;562933][URL="https://www.geeksinphoenix.com/blog/post/2014/01/12/How-to-tell-what-type-of-USB-connector-you-have-by-color.aspx"]USB 2.0 sockets are usually black and USB 3.0 blue.[/URL][/QUOTE]

Useful to know, thx.

Install seems to have gone fine, got to the "restart system now" prompt, upon which subsequent reboot got the expected Ubuntu - I paraphrase -"please remove install medium and press 'Enter'" prompt, did that, Ubuntu startup began, the little scrolling dots below the logo to effect of 'working...', but never got to the hoped-for login prompt, system is hung with black screen with mouse cursor frozen. No more time for this today, tips as to what to try tomorrow welcome.

paulunderwood 2020-11-11 23:08

[QUOTE=ewmayer;562938]
Install seems to have gone fine, got to the "restart system now" prompt, upon which subsequent reboot got the expected Ubuntu - I paraphrase -"please remove install medium and press 'Enter'" prompt, did that, Ubuntu startup began, the little scrolling dots below the logo to effect of 'working...', but never got to the hoped-for login prompt, system is hung with black screen with mouse cursor frozen. No more time for this today, tips as to what to try tomorrow welcome.[/QUOTE]

This sounds bad.

One thing to try if the keyboard is responding is Ctrl+Alt+F1 or Ctrl+Alt+F2 to get you to a login prompt. Usually it is Ctrl+Alt+F7 to get you back to the GUI.

If the keyboard is not responding, select a different image at boot time -- maybe a rescue mode one

Nick 2020-11-11 23:18

I don't use Ubuntu but in my Linux distribution you can press Escape during the boot dots graphic to get the old-fashioned boot messages on the console instead. which may be more helpful.

EdH 2020-11-12 01:56

[QUOTE=Nick;562941]I don't use Ubuntu but in my Linux distribution you can press Escape during the boot dots graphic to get the old-fashioned boot messages on the console instead. which may be more helpful.[/QUOTE]
This works in Ubuntu as well. Subsequent ESC presses toggle between the graphic and the text listing.

kriesel 2020-11-12 02:17

Sign of life
 
1 Attachment(s)
After a long unsatisfying day of trying to revive my network after a bad overnight storm
(it could be worse; the neighbor's new shed rolled nearly to their sun porch, uphill, and is looking very much the worse for its FOURTH excursion off its foundation), was inspired by Ernst's update and tempted fate further with Hydra + Windows. Probably should have gone with evaluating on "Windows Pro for Workstation" instead of "Windows Pro" for more cores & cache supported. Time stamp on attachment is PST from before correcting to my time zone. Elapsed time for OS install including rounding up media and an external USB DVD drive ~2 hours including nudging it a bit now and then while mostly doing other things.
Drive is partitioned so there is room left for a Linux install.
That's the good news.

Now the other kind.
Windows subsets the core count rather severely. If Windows for Workstation allows twice what Windows Pro does, that's still 56 out of 68. I assume mprime in WSL2 would not be able to bring any more cores to bear. In any event, it is impressive to see 112 logical processors represented in Resource Monitor at this point, especially given the low used hardware price.

Prime95 v30.3b6 generates this error in the application log, without ever making an appearance on the screen.[CODE]Faulting application name: prime95.exe, version: 30.3.1.0, time stamp: 0x5f5ae7c7
Faulting module name: libhwloc-15.dll, version: 0.0.0.0, time stamp: 0x5e820fc3
Exception code: 0xc0000005
Fault offset: 0x0000000000011cc8
Faulting process id: 0x30d0
Faulting application start time: 0x01d6b89b6e30c9ab
Faulting application path: C:\Users\ken\Documents\prime95\prime95.exe
Faulting module path: C:\Users\ken\Documents\prime95\libhwloc-15.dll
Report Id: e2331268-fb28-457e-adc7-288514d55ef5
Faulting package full name:
Faulting package-relative application ID: [/CODE]Haven't tried mlucas or mfactor yet.

kriesel 2020-11-12 03:25

1 Attachment(s)
CPUZ view of the Windows-flavor-subsetted hardware, and a benchmark on it.
Feature Detector output:[CODE]CPU Vendor String: GenuineIntel

CPU Vendor:
AMD = No
Intel = Yes

OS Features:
64-bit = Yes
OS AVX = Yes
OS AVX512 = Yes

Hardware Features:
MMX = Yes
x64 = Yes
ABM = Yes
RDRAND = Yes
RDSEED = Yes
BMI1 = Yes
BMI2 = Yes
ADX = Yes
MPX = No
PREFETCHW = No
PREFETCHWT1 = Yes
RDPID = No
GFNI = No
VAES = No

SIMD: 128-bit
SSE = Yes
SSE2 = Yes
SSE3 = Yes
SSSE3 = Yes
SSE4a = No
SSE4.1 = Yes
SSE4.2 = Yes
AES-NI = Yes
SHA = No

SIMD: 256-bit
AVX = Yes
XOP = No
FMA3 = Yes
FMA4 = No
AVX2 = Yes

SIMD: 512-bit
AVX512-F = Yes
AVX512-CD = Yes
AVX512-PF = Yes
AVX512-ER = Yes
AVX512-VL = No
AVX512-BW = No
AVX512-DQ = No
AVX512-IFMA = No
AVX512-VBMI = No
AVX512-VPOPCNTDQ = No
AVX512-4FMAPS = No
AVX512-4VNNIW = No
AVX512-VBMI2 = No
AVX512-VPCLMUL = No
AVX512-VNNI = No
AVX512-BITALG = No
AVX512-BF16 = No

Summary:
Safe to use AVX: Yes
Safe to use AVX512: Yes[/CODE]

ewmayer 2020-11-12 22:53

Thanks to Nick et al for the boot-diagnostic tips - I may not have further time to play with the KNL until the weekend, will first try the above and glean whatever diagnostics I can, will also dig out the old let's-buy-GIMPS-a-KNL thread and see what OS what installed on that. It may simply turn out to be an issue of KNL needing one of some specific subset of server-class Linux distros.

@Ken: Did you buy your Hydra from the same eBay vendor you pointed me to? Also note:

" If Windows for Workstation allows twice what Windows Pro does, that's still 56 out of 68. I assume mprime in WSL2 would not be able to bring any more cores to bear. In any event, it is impressive to see 112 logical processors represented in Resource Monitor"

-- IIRC the KNL is actually *four* logical processors for each physical one. If you're only seeing 2-per that is likely another limitation of your Windows version, but as you note w.r.to 56-of-68, codes which can actually make use of more than (say) 2 logical processors per physical one are likely rare.

kriesel 2020-11-12 23:55

[QUOTE=ewmayer;563039]Thanks to Nick et al for the boot-diagnostic tips - I may not have further time to play with the KNL until the weekend, will first try the above and glean whatever diagnostics I can, will also dig out the old let's-buy-GIMPS-a-KNL thread and see what OS what installed on that. It may simply turn out to be an issue of KNL needing one of some specific subset of server-class Linux distros.

@Ken: Did you buy your Hydra from the same eBay vendor you pointed me to?[/QUOTE]Yes, and first. Taking no chances.

[QUOTE]Also note:

" If Windows for Workstation allows twice what Windows Pro does, that's still 56 out of 68. I assume mprime in WSL2 would not be able to bring any more cores to bear. In any event, it is impressive to see 112 logical processors represented in Resource Monitor"

-- IIRC the KNL is actually *four* logical processors for each physical one. If you're only seeing 2-per that is likely another limitation of your Windows version, but as you note w.r.to 56-of-68, codes which can actually make use of more than (say) 2 logical processors per physical one are likely rare.[/QUOTE]Windows 10 Pro recognized 28 cores and 112 logical processors, so at the expected 4:1 ratio. See the Taskmgr screen shot that shows those numbers along with an unexpected claim of 2 sockets, at post 57, or the cpu-z left instance, bottom right portion at post 58. An install of Win 10 Pro for Workstations "MIGHT" double both those, and still fall short of the 68 actual cores, 272 logical that a 7250 implies, with license cost ~$300. Not sold on that bargain. Will try adding on WSL / Ubuntu, and bootable Ubuntu, as time allows.

My network remains recalcitrant, and currently I have some of it isolated, so that the rest functions. I suspect a jabbering NIC or bad switch lurking among the quarantined few items. Something goes awry that causes DHCP to fail and remote desktop sessions to fail and icmp ping to fail on the wireless especially. The quarantined stuff is all wired.

kriesel 2020-11-13 01:35

Updating to Windows 10 Pro build 1909 paid off
 
2 Attachment(s)
Now sees 1 socket, 68 real cores, 272 logical cores, and all cache.
And prime95 v30.3b6 successfully launches and recommends 17 4-core workers.
Benchmarking is under way for 2M to 8M on selected worker counts. Based on the CPUZ benchmarking and comparisons to other cpus, it will be faster than an i7-4790 but not as fast as recent chips.

kriesel 2020-11-15 11:44

[QUOTE=ewmayer;562938]but never got to the hoped-for login prompt, system is hung with black screen with mouse cursor frozen. No more time for this today, tips as to what to try tomorrow welcome.[/QUOTE]WSL2 install was a bust. The cpu lacks some required virtualization support.

On Windows, I have gotten to where it will run prime95. Benchmarking with hyperthreading in prime95 does x2, x3, and x4, which are almost always progressively slower.

Running higher prime95 benchmarking overnight, it has settled into a state after several hours where the screen is black except for a slowly responding white mouse cursor. My recollection is Windows was already configured not to put the system into standby or hibernation ever.
It is unresponsive at the console or by remote desktop, but responds to ping, and to the Numlock key slowly. This is close to what Ernst describes, but my system has not yet had Linux installed, and his has presumably not had Windows installed. Perhaps disabling hibernation in ACPI settings is needed, or a careful review of Windows power settings, or both.

My unit is wired... peculiarly. There are no connections to JF1 (which has pinouts for power switch, reset switch, power LED, HDD LED, NICs LEDs etc). The case-front power button has no effect.

The hardware seller's stock is now sold out.

kriesel 2020-11-15 16:41

Wasn't expecting this in a benchmark run; never seen it before. Occurrence of "INF".
First appearance at 18M and becomes more prominent as fft length increases. Some "15 second" benchmark timings take up to an hour on the Xeon Phi 7250 running prime95 v30.3b6.

In my unit's ACPI configuration in BIOS, there are only two lines, neither having any mention of or apparent relation to hibernation.

ewmayer 2020-11-18 20:09

Finally completed a bunch of proof-of-principle coding/testing re. the p-1 algo which will debut in Mlucas v20 yesterday, had a little time to play with the KNL I setup with Ubuntu 19.10 again - on boot, once it gets to the Ubuntu-load screen, htting Esc has no effect, it simply continues on to PSOD (purple screen of death) as before. Again, my expectation was that the KNL would need some enterprise/server flavor of Linux, possibly also a custom Intel software stack. Did a bit of digging:

o From the [url=https://mersenneforum.org/showthread.php?t=21544]Let's Buy GIMPS a KNL[/url] thread, post #43: "it comes preconfigured with all the tools and [url=https://en.wikipedia.org/wiki/CentOS]CentOS[/url]", said Linux distro being a free enterprise-oriented fork of Redhat, whose own enterprise distro, RHEL, requires a paid subscription.

o Intel's website, OTOH, describes the [url=https://software.intel.com/content/www/us/en/develop/articles/intel-manycore-platform-software-stack-mpss.html]Intel Manycore Platform Software Stack[/url]:
[quote]Manycore Platform Software Stack." is necessary to run the Intel® Xeon Phi Coprocessor. Users often call this stack "MPSS" for short. It is dependent on Linux kernels 2.6.34 or later , and it has been tested to work with specific versions of 64-bit Operating Systems:

o Red Hat Enterprise 6.0, 6.1, 6.2, 6.3, 6.4 and 6.5 (for MPSS 3.2 and earlier releases); versions 6.3, 6.4, 6.5, 6.6 and 7.0 (MPSS 3.3 and 3.4), versions 6.4, 6.5, 6.6 and 7.1 (MPSS 3.5), versions 6.7, version 7.2 (MPSS 3.6 and 3.7) and version 7.3 (MPSS 3.8)
o SuSE Linux Enterprise Server (SLES) 11 SP1 and SP2 (MPSS 2.1), SuSE Linux Enterprise Server (SLES) 11 SP2 and SP3 (MPSS 3.3), SuSE 11 SP2 and SP3, SuSE 12 (MPSS 3.4), SuSE 11 SP3 and SuSE 12 (MPSS 3.5), SuSE 11 SP4, SuSE 12 SP1 (MPSS 3.6 and 3.7), SuSE 12 SP2 (MPSS 3.8)
o Microsoft* Windows 7 Enterprise SP1, Windows 8/8.1 Enterprise, Windows 10, Windows Server 2008 R2 SP1, Windows Server 2012 and Windows Server 2012 R2

The readme files (referenced in the Download section) have more information on how to build and install the stack.

The open source updates we have made are in support of the instruction set, the ABI, initializing and controlling an SMP on-a-chip, and the glue software to support the coprocessor communication with the host system. The changes in the Linux kernel are primarily for three reasons:

o Numerous little changes to support the unique combination of an Intel® Pentium® processor core that also supports 64-bits including the Intel® Initial Many Core Instructions (Intel® IMCI).
o Power management, which is a feature not associated with the original Pentium processors. Power management is much more important when you have up to 61 cores on a single die
o The Intel® Many Integrated Core (MIC) check architecture, also a feature not present in the original Pentium processor designs.

The Symmetric Communications InterFace (SCIF) is included in the RPM bundle. SCIF provides a mechanism for inter-node communications within a single platform. A node, for SCIF purposes, is defined as either a Intel® Xeon Phi Coprocessor or the Intel® Xeon® processor. In particular, SCIF abstracts the details of communicating over the PCI Express bus. The SCIF APIs are callable from both user space (uSCIF) and kernel-space (kSCIF).[/quote]
Now if CentOS works as advertised, it should be drop-in-able in place of RHEL, right? And are all those Intel special tools likely must-haves, or nice-to-haves? (E.g. the SCIF sounds like something people with multiple Xeon processors might want or need, single-Xeon-ers like me less so.)

Xyzzy 2020-11-18 20:49

[QUOTE]For developers, getting Red Hat Enterprise Linux is now easier than ever thanks to the availability of the no-cost Red Hat Developer Subscription.[/QUOTE][URL]https://developers.redhat.com/articles/getting-red-hat-developer-subscription-what-rhel-users-need-know[/URL]

Xyzzy 2020-11-18 20:52

[QUOTE=ewmayer;563691]…Ubuntu 19.10 again - on boot, once it gets to the Ubuntu-load screen, htting Esc has no effect, it simply continues on to PSOD (purple screen of death) as before.[/QUOTE]Most likely you need to remove [C]quiet[/C] from the kernel command line. We are not sure how to do this with Ubuntu but we know it is possible. Once that is done you can see the boot message log to determine where the failure is.

ewmayer 2020-11-18 23:48

I prefer not having to online-register for anything if it can be avoided. Created boot disk using CentOS iso file (CentOS-8.2.2004-x86_64-minimal.iso, DLed from one of the official mirrors), plugged into the KNL and booted from it using the "install" option, got through a bunch of steps with a green "[ OK ]" in the terminal output, then hit a bunch of repeated warnings of form
[i]
[time-since-boot] dracut-initqueue[3265]: Warning: dracut-initqueue timeout - starting timeout scripts
[/i]
After a few minutes of that repeating, got
[code]
Starting Setup Virtual Console...
[ OK ] Started Setup Virtual Console.
Starting Dracut Emergency Shell...
Warning: /dev/root does not exist

Generating "/run/initramfs/rdsosreport.txt"
Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report.

dracut:/#[/code]
The journalctl command emulates Linux "more", which is unavailable in this emergency-mode shell environment. Some highlights - note this appears in fact to be a 68-physical-core (4 logical CPUs per physical one, thus 272) system like kriesel's, not the 64-core one I thought I was buying (not that I'm complaining, mind you :) - with a few annotations by me in []:
[code]
smpboot: Allowing 272 CPUs, 0 hotplug CPUs
...
Booting paravirtualized kernel on bare hardware.
...
Kernel command line: BOOT_IMAGE=vmlinux [stuff about the CentOS iso] quiet
Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware are certified with Red Hat Ent[line cuts off]
...
x86: Booting SMP configuration:
...
smp: brought up 1 node, 272 CPUs
...
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
ACPI: bus type PCI registered
...
ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[bunch of pci-init stuff]
SCSI subsystem initialized
...
can't derive routing for PCI INT D
PCI INT D: not connected
...
New USB device found [stuff re. Linux boot image]
...
can't derive routing for PCI INT D
PCI INT D: no GSI
...
[stuff about loading CentOS kernel signing key]
...
[this is in bright red font]usb 3-8: device descriptor read/64, error -110
...
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k
...
sd 3:0:0:0: [sda] Attached SCSI disk [preceding line confirm 1TB, i.e. the SSD I bought gor this build][/code]
Then we get the repeating "dracut" timeout warnings I mentioned above, ending in "Warning: Could not boot." and the Emergency-shell stuff.

paulunderwood 2020-11-19 02:57

This person claims to have "fixed the problem"

[url]https://forums.centos.org/viewtopic.php?t=63043[/url]

by running [C]dracut -f[/C] in rescue(?) mode. He might have meant emergency mode,

[url]https://linux.die.net/man/8/dracut[/url]

Another solution seems to be to enter "exit" at the dracut prompt. This might be a safer option to try firstly.

ewmayer 2020-11-19 03:57

@paul: Thanks for digging that out. Interestingly, it proved unnecessary - I powered the system back up with the boot USB plugged in, waited the several minutes this system needs to run all its BIOS stuff, then hit <f11> at the SuperMicro boot screen. Now for the new part - the first time I did the above, the boot menu listed 3 items correspondng to the boot USB: 'boot general' and under that 'boot partition 1' and 'boot partition 2'. I recalled that after creating the boot USB, dd copied the iso-image to partition 1, but on try #1 I just hit 'boot general'. This time I selected 'boot partition 1' and everything worked, root/user-info all entered and it's copying files and configuring the kernel as I write this. Fingers crossed, time to get dinner and go offline for the evening. Update tomorrow.

axn 2020-11-19 06:47

[QUOTE=Xyzzy;563697]Most likely you need to remove [C]quiet[/C] from the kernel command line. We are not sure how to do this with Ubuntu but we know it is possible. Once that is done you can see the boot message log to determine where the failure is.[/QUOTE]

[url]https://askubuntu.com/questions/477821/how-can-i-permanently-remove-the-boot-option-quiet-splash[/url]

EDIT:- Obviously you need to boot first to do this :-(

xilman 2020-11-19 11:44

[QUOTE=axn;563727][url]https://askubuntu.com/questions/477821/how-can-i-permanently-remove-the-boot-option-quiet-splash[/url]

EDIT:- Obviously you need to boot first to do this :-([/QUOTE]Yes, but not necessarily the system you wish to boot.

Extract the boot disk and plug into another Linux box which boots properly. Mount the added disk somewhere, edit to your heart's content, then do a clean shutdown, followed by re-inserting the disk into its own machine.

Done this several times when systems won't boot from their own disk and I don't have a convenient bootable CD/DVD/memstick to hand.

kriesel 2020-11-19 14:24

[QUOTE=ewmayer;563725]waited the several minutes this system needs to run all its BIOS stuff, then hit <f11> at the SuperMicro boot screen.[/QUOTE]
Some observations here:
1) Sometimes mine seems to get stuck during the BIOS initialization, requiring a power cycle to try again.
2) The sequence is interminable but the time window for F11, F12, or DEL to select options from the white SuperMicro boot screen is brief. It would be nice to be able to shorten the one or lengthen the other.
3) Haven't experimented with BIOS settings to possibly skip / disable some portions of the initialization.
4) The BIOS seems to support a commercial-size-kitchen-sink set of approaches. Disabling the unused ones might provide a considerable startup speedup, if possible, by eliminating timeout periods for things that ain't gonna happen (IPMI IP# issuance for example).
5) Jumper changes are another possibility. BMC disable.
6) One more way my system is wired oddly; documentation for the motherboard indicates the two adjacent RJ45 jacks are regular LAN ports, but if the one nearer the USB (#7 in fig 5-2 of the manual found online) is connected, DHCP fills in an IPMI IP# (remote console via IP), instead of providing LAN connectivity.

Some good news is checking prime.log and the worker windows of prime95 shows no sign of errors detected, in the 17 workers' 58.3M-59.1M LL DC progress, to 31-35% each so far and a few Jacobi checks each. These should all complete by about month's end.

ewmayer 2020-11-19 20:56

Install last night finished successfully and after reboot from the installation-on-SSD we got a login prompt, but in basic-terminal mode ... must've overlooked whatever option is needed to install the windowing system in the initial-install screens. So did 'shutdown -h now', just now replugged in the boot USB ... and the system won't power up. Note, the front-panel power button on this is mis-connected, if it's connected at all - have always simply needed to unplug/replug the power cord in the back to turn it off/on, until successful CentOS install last night at least gave us the 'shutdown -h now' option for the 'off' part. Grrr ... no time right now to poke around in the damn case and wiretrace and whatnot. I was hoping to be building and testing code on this sucker by now ... annoying as hell.

kriesel 2020-11-19 21:48

[QUOTE=ewmayer;563788] ... and the system won't power up. Note, the front-panel power button on this is mis-connected, if it's connected at all - have always simply needed to unplug/replug the power cord in the back to turn it off/on, until successful CentOS install last night at least gave us the 'shutdown -h now' option for the 'off' part. Grrr ... no time right now to poke around in the damn case and wiretrace and whatnot. I was hoping to be building and testing code on this sucker by now ... annoying as hell.[/QUOTE]Unplug, open case, check [URL="https://www.mersenneforum.org/showpost.php?p=563263&postcount=62"]JF1[/URL] by the main motherboard 24-pin power connector. I had the same won't-start issue after a Windows shutdown -s. But -r (restart) was not typically a problem. It certainly created a sinking feeling. And after that the box was completely unresponsive including to power strip cycling; no POST, no signal at the VGA, most onboard LEDs stayed off, radiator fans did not spin up to mini-propjet-takeoff-sound-level initially, as they previously had. After a while I managed to recover mine by putting a separate power button on JF1 temporarily. Then reconnect power, push the added button, go to BIOS settings, change Power On behavior, from Last State, to Always Start, to reduce the occurrence of repeats.
It's a catch22; system won't restart with power because its last state was off, because that's what the user told it to be through the OS; can't turn it on because the case power switch is not connected; can't change the BIOS because it won't turn on, because...

This case has a sort of "secret compartment". One side panel removed gives a view of cpu, pcie, motherboard-top-side etc and 3.5" drive mountings. The opposite side removed gives 2.5" drive mountings and a view of some cabling. In mine I found an unconnected end of the power sw, reset sw, power LED, drive LED cable in that "secret compartment". So you might be able to activate the front power button by fishing that out and connecting "Power sw" at JF1 pins 1 & 2, if you don't have a spare switches/LEDs cable assembly in your parts drawer.

This thing is like a sports car (or some partners I've known). Expect some cost/pain as the price of the interesting or fun times.
I suspect these were set up to run as part of a server farm, in rows of warehouse style welded-wire racks, with Unit ID LEDs and remote management enabled, local intervention disabled.

Congrats on getting over the OS install hurdles thus far, and thanks for confirming the hardware obstinacy as shipped. I'm wondering how minimal is CentOS -minimal, at 1.6GB. GUI included? The other iso is 7.7GB, too big for a DVD.

FYI prime95 benchmark results on Win10 can be found at [URL]https://www.mersenneforum.org/showthread.php?p=563304#post563304[/URL]. A large screen or a magnifier or some serious zoom may be useful.

ewmayer 2020-11-20 20:13

2 Attachment(s)
@above: Thanks, ken, you're a lifesaver! (Or at least a major time saver).

OK, removed the sidepanel under bottom side of mobo, found the dangling power+led connector bundle. The 2-pin power one has no color coding, all black, no +- polarity marking, just "Power SW" label on one side. I found a downloadable mobo PDF at the [url=https://www.manualslib.com/manual/1406892/Supermicro-5038k-I.html]Manualslib site[/url] I linked to in html-manual form previously, and have attached a snip of the mobo section in question. Say I'm looking at the mobo at the same orientation - big 24-pin main power plug at bottom, smaller 12-pin jumper array above it, 2 power pins at far [strike]left[strike]right end of the latter. Do I want to hook the 2-pin power-button connector to the latter pair of pins so that the "Power SW" label faces leftward or rightward?

Nick 2020-11-20 21:35

Is there no CLR CMOS jumper on the mobo any more these days?

PhilF 2020-11-20 21:38

[QUOTE=ewmayer;563865]Do I want to hook the 2-pin power-button connector to the latter pair of pins so that the "Power SW" label faces leftward or rightward?[/QUOTE]

For the power switch the polarity doesn't matter.

kriesel 2020-11-20 23:04

[QUOTE=ewmayer;563865]Do I want to hook the 2-pin power-button connector to the latter pair of pins so that the "Power SW" label faces leftward or rightward?[/QUOTE]Don't know. There may be a reason why it was hidden away like that. Trace if possible. Or get something like [URL]https://www.ebay.com/itm/Desktop-Computer-Case-ATX-Power-Reset-2-Switch-Cable-HDD-Power-LED-Lights/221545392426[/URL] You don't have to use it all.
Did you by chance buy the warranty?

kriesel 2020-11-20 23:06

[QUOTE=Nick;563880]Is there no CLR CMOS jumper on the mobo any more these days?[/QUOTE]Manual says there's a pad pair to short with a screwdriver blade with power off.

ewmayer 2020-11-21 03:12

[QUOTE=PhilF;563881]For the power switch the polarity doesn't matter.[/QUOTE]

That was my thinking, as well - since one pin is PWR and the other GND, pressing the (once-wired-up) "on" button should simply connect the 2, but figured better safe than sorry, ask the experts first. Now if I blow myself up, I can blame you from the aferlife, or something. :)

Anyhow, was busy with code for rest of day, so will wait 'til tomorrow so can do by daylight.

[b]Edit:[/b] LOL, glad I decided to wait! Just added a second attachment to my post #75, showing the Control Panel with the individual pins labeled. In that, PWR and its corresponding GND are at top, but the 2 pins marked unused via 'X' are the next-to-last pin pair at the opposite end of the array. That means the pin highlighted via the solid black square in the first attachment is the PWR pin, the one the Control Panel diagram text says should be connected to the red wire in the non-existent-in-my-case ribbon-cable bundle. So it still shouldn't matter which way the labeled flat side of my 2-pin hookup faces, but it needs to connect to the rightmost pin pair in the corresponding pinout array as pictured in the first ,'pwrconnects', attachment in post #75.

ewmayer 2020-11-21 21:10

Success! Had to unplug the SSD and the 24-pin main-power plug to be able to plug in the little 2-pin power-switch connector, once everything was hooked back up and power cord plugged in, power button worked as intended.

Redid the CentOS install from the boot USB, this time added several optional packages including the Gnome desktop and dev-tools, told the installer to reclaim all the space on the SSD used by the older install, no problems.

But - and there's always a but, it seems - once install done, pulled boot USB, clicked "Done" to reboot, <f11> to now use the CentOS install on the SSD, again ended with a basic terminal-mode login prompt, no signs of a Desktop GUI. Did command-shell login, "which gcc" came up empty, so either the dev-tools package does not include it, or something else is awry. Shutdown for now, no more time for this today.

paulunderwood 2020-11-21 22:18

[QUOTE=ewmayer;563972]Success! Had to unplug the SSD and the 24-pin main-power plug to be able to plug in the little 2-pin power-switch connector, once everything was hooked back up and power cord plugged in, power button worked as intended.

Redid the CentOS install from the boot USB, this time added several optional packages including the Gnome desktop and dev-tools, told the installer to reclaim all the space on the SSD used by the older install, no problems.

But - and there's always a but, it seems - once install done, pulled boot USB, clicked "Done" to reboot, <f11> to now use the CentOS install on the SSD, again ended with a basic terminal-mode login prompt, no signs of a Desktop GUI. Did command-shell login, "which gcc" came up empty, so either the dev-tools package does not include it, or something else is awry. Shutdown for now, no more time for this today.[/QUOTE]

[url]https://www.golinuxcloud.com/install-gnome-enable-gui-mode-rhel-centos-8/[/url]

ewmayer 2020-11-21 22:30

[QUOTE=paulunderwood;563977][url]https://www.golinuxcloud.com/install-gnome-enable-gui-mode-rhel-centos-8/[/url][/QUOTE]

"On CentOS 8 you do not any additional subscription, you just need an active internet connection to be able to download the GNOME Desktop Environment packages."

I did the server-with-gui install ... woulda been nice if the installer had mentioned the above requirement.

Bit of a catch-22 ... for my Ubuntu installs, the GUI was immediately available, and I used its interactive options to quickly cofigure a Wifi stick. Should I have plugged that in at initial-install time and configured it using the networking menu in the CentOS installer? (Re. the plugging-in, this mobo has just 2 back-panel USB 2 ports, which are all used in install - kybd, mouse, install USB - but hopefully, unlike the CentOS install USB which needs USB2, my wifi stick can use one of the front-panel USB3 ports.)

Would the "workstation" install type behave more like I'm used to with Ubuntu?

paulunderwood 2020-11-21 22:46

[QUOTE=ewmayer;563978]"On CentOS 8 you do not any additional subscription, you just need an active internet connection to be able to download the GNOME Desktop Environment packages."

I did the server-with-gui install ... woulda been nice if the installer had mentioned the above requirement.

Bit of a catch-22 ... for my Ubuntu installs, the GUI was immediately available, and I used its interactive options to quickly cofigure a Wifi stick. Should I have plugged that in at initial-install time and configured it using the networking menu in the CentOS installer? (Re. the plugging-in, this mobo has just 2 back-panel USB 2 ports, which are all used in install - kybd, mouse, install USB - but hopefully, unlike the CentOS install USB which needs USB2, my wifi stick can use one of the front-panel USB3 ports.)

Would the "workstation" install type behave more like I'm used to with Ubuntu?[/QUOTE]

I don't think so, but it might be worth a try -- you should be using workstation??

Try, as per instructions:

[C]dnf grouplist[/C]

you should see "Server with GUI" if that concurs with what you have installed,

Since you have Gnome installed, the next thing to do is:

[C]systemctl list-units --type target --all | egrep graphical.target[/C]

[C]systemctl isolate graphical.target[/C]

[C]systemctl list-units --type target --all | egrep graphical.target[/C]

Then do the commands in "Change default target to start RHEL/CentOS 8 Server with GUI at reboot"

HTH

PhilF 2020-11-21 23:46

Do you really need the GUI anyway? Unless you have specific applications that require it, there isn't anything you can do in the GUI that can't be done in a terminal window.

I never do install any GUI interfaces when doing a linux install (which has always been Debian, but the latest ROCm drivers insist on Ubuntu this time).

paulunderwood 2020-11-21 23:51

[QUOTE=PhilF;563980] (which has always been Debian, but the latest ROCm drivers insist on Ubuntu this time).[/QUOTE]

You can specify an older driver in the /etc/apt/source.list.d/rocm.list file by changing "debian" to a version number.

ewmayer 2020-11-22 01:09

[QUOTE=paulunderwood;563979]I don't think so, but it might be worth a try -- you should be using workstation??

Try, as per instructions:

[C]dnf grouplist[/C]

[snip ... IDK how you remember all that crap :][/QUOTE]

Figured 'server' might enable some useful admin options, should they prove necessary later ... thought '+gui' meant install-from-boot-image, a la Ubuntu. Annoying that the installer, since it obviously was running sans net connection, remained silent about that.

Anyhow, above cmd fives errors which again translate to "you gots to be on the interwebs":
[i]
CentOs-8 AppStream
Errors during downloading metadata for repsitory 'AppStream'
- Curl error (6): Couldn't resolve hostname for [url]http://mirrorlist.centos.org/[/url],,, etc.[/i]

@PhilF: GUI may not be *necessary*, I just find it much easier to do many tasks that way - again, I'm not the type of person who can rote-memorize complex command sequences like PaulU apparently is. There's a reason the GUI was invented, we should not be slaves to it, but same goes to command-line mode. The way I picture things is via the common word "Desktop" to refer to GUI: command-line mode is like trying to do office work with the only usable space being a small rectangle atop one's filing cabinet sufficient to hold 1 or 2 sheets of paper. Everything one *needs* is in said cabinet, sure, but managing multiple tasks which are not all monotasking-mode completable is a nightmare.

Gonna try one more round of boot-from-usb-stick, this time 'workstation' setup and with wifi stick plugged into one of the front-panel USB3 ports. Update tomorrrow, and thx again for all the guidance.

Worst-case, how much hassle it is to configure a wifi stick in cmd-line mode? Ubuntu had no problem finding a suitable driver for the same brand of stick in the install repository, i.e. w/o needing to DL any custom drivers over the (nonexistent, because that's what the wifi stick is for) internet connection.

ewmayer 2020-11-22 02:10

OK, some progress, quick update then offline 'til tomorrow:

With wifi stick inserted into usb3 slot @boot-from-media time, got this scary-looking but as-it-proved nonfatal error:

[i] dracut-pre-udev[2742]: modprobe: ERROR: could not insert 'edd' / usb3-3: device descriptor error
[/i]
Note: For some reason whenever I try a fresh boot-from-usb-image, 1st try ends in same 'dracut timeuot' error I mentioned for my initial install try a couple days ago. 'exit' does nothing useful, but ctrl-alt-del and retry-with-identical-boot-from-option-via-<f11> works ... no idea why this 2-try, but hopefully it's behind me now.

The 'base environment' options at boot-from-usb don't include workstation, they are these 3:

o Server
o Minimal install - that alas causes 'GNOME' to vanish from the neighboring 'additional software' pane
o Custom operating system - yah, that sounds not-for-me

Again chose server+gui, this time network-setup showed the wifi and was able to password-connect to that - yay!

Post-install, again rebooted-from-new-CentOS-kernel-on-SSD, again got cmd-line mode (expected), but it seems wifi is not auto-enabled even though I configured it during install and it clearly was web-connected (e.g. internet time stuff showed up). 'dnf grouplist' gives same no-connecttion error or before. So back to same issue: how to enable wifi from cmd-line. Hopefully easy now that it's been properly preconfigured.

paulunderwood 2020-11-22 08:05

See this link: [url]https://www.krizna.com/centos/setup-network-centos-7/[/url]

I am running Centos8 under qemu installed from the same iso...

I ran:

[C]nmcli d[/C] to get a list of "nic cards".

As root, I edited (with nano) [C]/etc/sysconfig/network-scripts/ifcfg-<your device> [/C]and changed to [C]ONBOOT=yes[/C]. Then I ran [C]systemctl reboot[/C]. I now have internet access.

Next follow the link: [url]https://www.golinuxcloud.com/install-gnome-enable-gui-mode-rhel-centos-8/[/url] to install gnome, turn it on and make default at boot.

Run [C]systemctl reboot[/C] et voilà :smile:

ewmayer 2020-11-22 20:32

[QUOTE=paulunderwood;563994]See this link: [url]https://www.krizna.com/centos/setup-network-centos-7/[/url]

I am running Centos8 under qemu installed from the same iso...

I ran:

[C]nmcli d[/C] to get a list of "nic cards".

As root, I edited (with nano) [C]/etc/sysconfig/network-scripts/ifcfg-<your device> [/C]and changed to [C]ONBOOT=yes[/C]. Then I ran [C]systemctl reboot[/C]. I now have internet access.[/QUOTE]
Found the wifi stick in the output of [C]nmcli d[/C], did 'cat' on its ifcfg-file, the corresponding [c]NAME=[/c] entry shows the name of my internet router, and [C]ONBOOT=yes[/C] is already set. But e.g. 'ping yahoo.com' gives "Name or service not known" - still no web access. And 'ifconfig' does not show an entry for the wifi, which device name is wlp0s20u3.

[quote]Next follow the link: [url]https://www.golinuxcloud.com/install-gnome-enable-gui-mode-rhel-centos-8/[/url] to install gnome, turn it on and make default at boot.

Run [C]systemctl reboot[/C] et voilà :smile:[/QUOTE]

Did the steps to make [c]graphical.target[/c] the default, rebooted, but again got command-line mode. After logging in, systemctl shows [c]graphical.target[/c] loaded and active. But still no GUI, still no internet access. I notice the above page next has a section "Install KDE Environment" - do I need that, as well?

Nick 2020-11-22 20:36

If I have trouble with X, I sometimes start it manually with "startx" from a text terminal, and then check the log to see what happened.

paulunderwood 2020-11-22 20:47

[QUOTE=ewmayer;564050]Found the wifi stick in the output of [C]nmcli d[/C], did 'cat' on its ifcfg-file, the corresponding [c]NAME=[/c] entry shows the name of my internet router, and [C]ONBOOT=yes[/C] is already set. But e.g. 'ping yahoo.com' gives "Name or service not known" - still no web access. And 'ifconfig' does not show an entry for the wifi, which device name is wlp0s20u3.



Did the steps to make [c]graphical.target[/c] the default, rebooted, but again got command-line mode. After logging in, systemctl shows [c]graphical.target[/c] loaded and active. But still no GUI, still no internet access. I notice the above page next has a section "Install KDE Environment" - do I need that, as well?[/QUOTE]

Try this command as root: nmtui and "activate" the wireless device. ([URL="https://www.techrepublic.com/article/how-to-easily-edit-a-network-connection-on-a-centos-7-minimal-installation/"]Ref[/URL]). But it sounds like it is activated -- see below.

You won't get Gnome until it is downloaded! No need for KDE.

"Name or service not known" is a DNS problem. It appears to not be picking up your router's DNS servers (if it has any) -- with nmtui add this DNS server 1.1.1.1 under "edit a connection".

After you have Gnome set up you can run [c]dnf groupinstall "Development Tools" -y[/c]

paulunderwood 2020-11-23 01:48

Alternatively, just add [C]DNS1=1.1.1.1[/C] to ifcfg-wlp0s20u3 and then run [C]systemctl restart NetworkManager.service[/C]

You should now be able to ping yahoo.com and be able to install "Server with GUI" and "Development Tools"

If not, with troubleshooting in mind, can you:
[LIST=1][*]ping 127.0.0.1?[*]ping your router?[*]ping 1.1.1.1?[/LIST]

Xyzzy 2020-11-23 13:29

Could this be a [C]wpa_supplicant[/C] issue?

ewmayer 2020-11-23 20:07

[QUOTE=paulunderwood;564052]Try this command as root: nmtui and "activate" the wireless device. ([URL="https://www.techrepublic.com/article/how-to-easily-edit-a-network-connection-on-a-centos-7-minimal-installation/"]Ref[/URL]). But it sounds like it is activated -- see below.

You won't get Gnome until it is downloaded! No need for KDE.

"Name or service not known" is a DNS problem. It appears to not be picking up your router's DNS servers (if it has any) -- with nmtui add this DNS server 1.1.1.1 under "edit a connection".[/QUOTE]

Tried that - menu shows the wifi, inside "Edit a connection" for that looks OK, though I tried toggling the "show password" checkbox and couldn't find any key that did that, and the above page doesn't mention that, nor does it say anything about the possible special-casing needed for wifi vs ethernet. But I checked the wifi password at setup time - recall that during my last OS-install I was able to configure the wifi, initially entered the password incorrectly but the installer alerted me to that, it clearly was correctly talking to the router at that point.

Back to the Edit pane for the wifi: I see blank entries for the BSSID, Cloned MAC address and MTU fields, can you see what those look like for the wifi connection on your CentOS-running system?

So I didn't actually change any of the fields, just toggled to <OK>, which took me back to select-connection pane, hit <Back> there and switched to "Activate a connection", and the wifi doesn't show up there, so something is clearly missing from that device's setup.

[QUOTE=paulunderwood;564067]Alternatively, just add [C]DNS1=1.1.1.1[/C] to ifcfg-wlp0s20u3 and then run [C]systemctl restart NetworkManager.service[/C]

You should now be able to ping yahoo.com and be able to install "Server with GUI" and "Development Tools"[/QUOTE]

Did that - I needed to 'su' to append the DNS1 entry to the file, 'sudo' didn't work for that - no joy.

[QUOTE]If not, with troubleshooting in mind, can you:
[LIST=1][*]ping 127.0.0.1? yes[*]ping your router? Network is unreachable (IP = 192.168.0.1, I first checked it via ping from my macbook)[*]ping 1.1.1.1? Network is unreachable[/LIST][/QUOTE]

paulunderwood 2020-11-23 23:57

[QUOTE=ewmayer;564125]

Back to the Edit pane for the wifi: I see blank entries for the BSSID, Cloned MAC address and MTU fields, can you see what those look like for the wifi connection on your CentOS-running system?

[/QUOTE]

Unfortunately I don't have wifi to test it out.

I think we are at a chicken and eggs situation. You might need to install wpa_supplicant as suggest by Xyzzy. This impasse will be overcome by using a LAN cable to connect to the net.

There are two last thing to try ([URL="https://unix.stackexchange.com/questions/603994/connect-to-wi-fi-with-minimal-installation-of-centos"]ref[/URL]):

nmcli device wifi
nmcli device wifi connect [ssid_name] password [type password]

BTW, the most important keys to use in nmtui are Tab, Space (as a selector), Cursor Keys and Enter,

Xyzzy 2020-11-24 03:49

Ernst: Does your wifi (AP) authenticate via WPA2 or WEP?


All times are UTC. The time now is 19:39.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.