[gpfsug-discuss] [External] Re: DSS-G upgrade and ConnectX-4 settings

[Jonathan Buzzard]

On Sat, 2022-09-10 at 20:50 +0000, Ryan Novosielski wrote:

[SNIP]

> It’s useful to hear these experiences before our site does this this
> month. What sort of DSS-G hardware do you have?
> 

We have a version 2 DSS-G system based on SR650's. It's a G2x0 style
system so two SR650's with ConnectX-4 cards and D3284 shelves.

> As far as the change of the networking interfaces, but believe that
> is something that happened in a relatively recent version of OFED,
> 5.2 to 5.4 maybe? It’s in the release notes anyhow. That bit me once
> on a different system, and I rolled back until I could plan for that
> better.
> 
> However, DSS-G installs have always used OFED, and GSS before them,
> not the RHEL-supplied drivers. I am as close to 100% confident about
> that as I am with anything.

You are correct DSS-G have always used OFED drivers. However the
interface name change between stock drivers and OFED drivers that now
exists throws a massive spanner in the works.

Basically it means you can't have xcat (or anything else for that
matter) setup the networking at install time as you could previously
which sucks.

I spent much of Sunday developing a shell script that nukes the network
configuration from on top and sets it all up using nmcli. However it
can't be run till after the install, and even then I have to put the
ports back into ethernet mode and reboot first.

As time passes I grow nostaglic for the days when everything was ethX.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG