[gpfsug-discuss] GPFS systemd and gpfs.gplbin

Simon Thompson S.J.Thompson at bham.ac.uk
Thu Jun 10 17:13:46 BST 2021


We manage kernel updates pretty carefully ... not least because there is a good chance MOFED will also break at the same time.

We do have a similar systemd unit that tries to install from our local repos, then tries to build locally.

Simon

On 10/06/2021, 15:01, "gpfsug-discuss-bounces at spectrumscale.org on behalf of Ryan Novosielski" <gpfsug-discuss-bounces at spectrumscale.org on behalf of novosirj at rutgers.edu> wrote:

    The problem with not version locking the kernel, however, is that you really need to know that the kernel you are going to is going to support the GPFS version that you are going to be running. Typically that only becomes a problem when you cross a minor release boundary on RHEL-derivatives, but I think not always. But I suppose you can just try this on something first just to make sure, or handle it at the repository level, or something else.

    Sent from my iPhone

    > On Jun 10, 2021, at 09:48, Skylar Thompson <skylar2 at uw.edu> wrote:
    > 
    > Thanks, Jonathan, I've been thinking about how to manage this as well and
    > like it more than version-locking the kernel.
    > 
    >> On Wed, Jun 09, 2021 at 09:28:07PM +0100, Jonathan Buzzard wrote:
    >> 
    >> So you need to apply a kernel update and that means a new gpfs.gplbin :-( So
    >> after going around the houses with several different approaches on this I
    >> have finally settled on what IMHO is a most elegant method of ensuring the
    >> right gpfs.gplbin version is installed for the kernel that is running and
    >> thought I would share it.
    >> 
    >> This is assuming you don't like the look of the compile it option IBM
    >> introduced. You may well not want compilers installed on nodes for example,
    >> or you just think compiling the module on hundreds of nodes is suboptimal.
    >> 
    >> This exact version works for RHEL and it's derivatives. Modify for your
    >> preferred distribution. It also assumes you have a repository setup with the
    >> relevant gpfs.gplbin package.
    >> 
    >> The basics are to use the "ExecStartPre" option of a unit file in systemd.
    >> So because you don't want to be modifying the unit file provided by IBM
    >> something like the following
    >> 
    >> mkdir -p /etc/systemd/system/gpfs.service.d
    >> echo -e "[Service]\nExecStartPre=-/usr/bin/yum --assumeyes install
    >> gpfs.gplbin-%v" >/etc/systemd/system/gpfs.service.d/install-module.conf
    >> systemctl daemon-reload
    >> 
    >> How it works is that the %v is a special systemd variable which is the same
    >> as "uname -r". So before it attempts to start gpfs, it attempts to install
    >> the gpfs.gplbin RPM for the kernel you are running on. If already installed
    >> this is harmless and if it's not installed it gets installed.
    >> 
    >> How you set that up on your system is up to you, xCAT postscript, RPM
    >> package, or a configuration management solution all work. I have gone for a
    >> very minimal RPM I call gpfs.helper
    >> 
    >> We then abuse the queuing system on the HPC cluster to schedule a "admin"
    >> priority job that runs as soon as the node becomes free, which does a yum
    >> update and then restarts the node.
    >> 
    >> 
    >> JAB.
    >> 
    >> -- 
    >> Jonathan A. Buzzard                         Tel: +44141-5483420
    >> HPC System Administrator, ARCHIE-WeSt.
    >> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
    >> _______________________________________________
    >> gpfsug-discuss mailing list
    >> gpfsug-discuss at spectrumscale.org
    >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
    > 
    > -- 
    > -- Skylar Thompson (skylar2 at u.washington.edu)
    > -- Genome Sciences Department (UW Medicine), System Administrator
    > -- Foege Building S046, (206)-685-7354
    > -- Pronouns: He/Him/His
    > _______________________________________________
    > gpfsug-discuss mailing list
    > gpfsug-discuss at spectrumscale.org
    > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
    _______________________________________________
    gpfsug-discuss mailing list
    gpfsug-discuss at spectrumscale.org
    http://gpfsug.org/mailman/listinfo/gpfsug-discuss



More information about the gpfsug-discuss mailing list