[gpfsug-discuss] memory needed for gpfs clients
Christopher Black
cblack at nygenome.org
Tue Dec 1 19:07:58 GMT 2020
We tune vm-related sysctl values on our gpfs clients.
These are values we use for 256GB+ mem hpc nodes:
vm.min_free_kbytes=2097152
vm.dirty_bytes = 3435973836
vm.dirty_background_bytes = 1717986918
The vm.dirty parameters are to prevent NFS from buffering huge amounts of writes and then pushing them over the network all at once flooding out gpfs traffic.
I'd also recommend checking client gpfs parameters pagepool and/or pagepoolMaxPhysMemPct to ensure you have a reasonable and understood limit for how much memory mmfsd will use.
Best,
Chris
On 12/1/20, 1:32 PM, "gpfsug-discuss-bounces at spectrumscale.org on behalf of Renata Maria Dart" <gpfsug-discuss-bounces at spectrumscale.org on behalf of renata at slac.stanford.edu> wrote:
Hi, some of our gpfs clients will get stale file handles for gpfs
mounts and it seems to be related to memory depletion. Even after the
memory is freed though gpfs will continue be unavailable and df will
hang. I have read about setting vm.min_free_kbytes as a possible fix
for this, but wasn't sure if it was meant for a gpfs server or if a
gpfs client would also benefit, and what value should be set.
Thanks for any insights,
Renata
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!C6sPl7C9qQ!H08HlNmBIkQRBOJKSHohzKHL6r39gAhQ3XTTczWoSmvffRFmQMcpJo8OyjMP7j-g$
________________________________
This message is for the recipient’s use only, and may contain confidential, privileged or protected information. Any unauthorized use or dissemination of this communication is prohibited. If you received this message in error, please immediately notify the sender and destroy all copies of this message. The recipient should check this email and any attachments for the presence of viruses, as we accept no liability for any damage caused by any virus transmitted by this email.
More information about the gpfsug-discuss
mailing list