[gpfsug-discuss] Adding client nodes using a shared NFS root image.

[Peter Childs]

We used to run

mmsdrestore -p manager -R /usr/bin/scp

in a xcat postscript to re-add our nodes to our Spectrum Scale cluster. however we disliked needing to put the private key for the whole cluster on every host,

We now use

mmsdrestore -N nodename

post-install from a management node to re-add the node to the cluster, so we could stop xcat from distributing the private key for security reasons.

Ideally we would have like the postscript call a manual call back to do this but have not as yet worked out how best to do this in xcat, so currently its a manual task which is fine when our nodes are stateless, but is not possible when your nodes are stateless.

My understanding is that xcat should have a hook to do this like the pre-scripts to run one at the end but I'm yet to find it.

Peter Childs

________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Ruffner, Scott (jpr9c) <jpr9c at virginia.edu>
Sent: Friday, January 29, 2021 8:04 PM
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] Adding client nodes using a shared NFS root image.

Thanks David! Slick solution.

--
Scott Ruffner
Senior HPC Engineer
UVa Research Computing
(434)924-6778(o)
(434)295-0250(h)
sruffner at virginia.edu


From: <gpfsug-discuss-bounces at spectrumscale.org> on behalf of "david_johnson at brown.edu" <david_johnson at brown.edu>
Reply-To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: Friday, January 29, 2021 at 2:52 PM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Adding client nodes using a shared NFS root image.

We use mmsdrrestore after the node boots. In our case these are diskless nodes provisioned by xCAT.  The post install script takes care of ensuring infiniband is lit up, and does the mmsdrrestore followed by mmstartup.
  -- ddj
Dave Johnson


On Jan 29, 2021, at 2:47 PM, Ruffner, Scott (jpr9c) <jpr9c at virginia.edu> wrote:
Hi everyone,

We want all of our compute nodes (bare metal) to directly participate in the cluster as client nodes; of course, they are sharing a common root image.

Adding nodes via the regular mmaddnode (with the dsh operation to replicate files to the clients) isn’t really viable, but if I short-circuit that, and simply generate the /var/mmfs/gen files and then manually copy those and the keyfiles to the shared root images, is that safe?

Am I going about this the entirely wrong way?

--
Scott Ruffner
Senior HPC Engineer
UVa Research Computing
(434)924-6778(o)
(434)295-0250(h)
sruffner at virginia.edu

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss