[gpfsug-discuss] GPFS diskless/stateless clients (was: Re: gpfs client and turning off swap)

[Ryan Novosielski]

We run stateless machines today with CentOS 7.x and have run GPFS 4.1.0.8, 4.2.3.x, and 5.0.x. We don’t use swap either in most cases, and have never needed to do anything special for that. We’ve generally needed to rig ExecStartPre stuff in systemd in order to make sure that the machines could restore their cluster membership.

If anyone else can share how they do that, it might be interesting information. In our case, this works (our provisioning system writes these variables as the actual host names) — we used to put this in an override to gpfs.service; I believe mmautoload.service is relatively new (either in 4.2.3.x to 5.0.x or 4.1.x to 4.2.x).

[root at test mmautoload.service.d]# pwd
/etc/systemd/system/mmautoload.service.d

[root at test mmautoload.service.d]# cat override.conf 
[Unit]
Before=slurmd.service
After=sys-subsystem-net-devices-ib0.device

[Service]
ExecStartPre=/usr/lpp/mmfs/bin/mmsdrrestore -p $CLUSTERMGR -R /usr/bin/scp
ExecStartPre=/usr/lpp/mmfs/bin/mmauth genkey propagate -N $NODENAME

--
____
|| \\UTGERS,  	 |---------------------------*O*---------------------------
||_// the State	 |         Ryan Novosielski - novosirj at rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
     `'

> On Nov 8, 2019, at 11:37 AM, Damir Krstic <damir.krstic at gmail.com> wrote:
> 
> Thanks - that's what I thought. We also (many refreshes ago) ran diskless images with RedHat 6 and GPFS 3.4 and ran into no issues with having swap off. I figured to ask in case something has changed with the Spectrum Scale. 
> Damir
> 
> On Fri, Nov 8, 2019 at 10:31 AM <david_johnson at brown.edu> wrote:
> We have most of our clients network booted and diskless — no swap possible.  Gpfs still works until someone runs the node out of memory....  
> 
>   -- ddj
> Dave Johnson
> 
> > On Nov 8, 2019, at 11:25 AM, Damir Krstic <damir.krstic at gmail.com> wrote:
> > 
> > 
> > I was wondering if it's safe to turn off swap on gpfs client machines? we have a case where checkpointing is swapping and we would like to prevent it from doing so by disabling swap. However, the gpfs manual admin. manual states to have swap enabled and sufficiently large, but it does not distinguish between clients and IO servers. 
> > 
> > Let me know when you get a chance.
> > 
> > Thank you.
> > Damir
> > _______________________________________________
> > gpfsug-discuss mailing list
> > gpfsug-discuss at spectrumscale.org
> > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss