<font size=2 face="sans-serif">Hi Phillip, </font><br><br><font size=2 face="sans-serif">this really depends on your application
workload. </font><br><font size=2 face="sans-serif">If your 2 node NSD cluster is the storage
cluster for some remote cluster(s), you probably don't need such a high
MaxFilesToCache value (nor a similar high maxStatCache value) at all.</font><br><br><font size=2 face="sans-serif">If there is an application workload
on that cluster as well, and this is mainly working on a high number of
files (like a login node for hpc applications of numerous users would do)
a high number of maxStatCache would only waste these 500 bytes (x 3000000)
(even if that is just 1.5GB) since the information is in the maxFilesToCache
entry as well. </font><br><br><font size=2 face="sans-serif">If your node however does loads of metadata
scans, inode traversal work ... mmbackup, policy runs, then a high
number for maxStatCache would be beneficial. </font><br><font size=2 face="sans-serif">John's (or Tomer's) presentation on
memory usage gives details on what the memory footprint of one or the other
would look like. </font><br><br><font size=1 face="Arial"><br>Mit freundlichen Grüßen / Kind regards</font><p><font size=2 face="Arial"><b>Achim Rehor</b></font><p><table width=600 style="border-collapse:collapse;"><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center></div><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=103 rowspan=10 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=right></div><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center></div><tr height=8><td width=52 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=199 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=245 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=52 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=199 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=245 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=52 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=199 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=245 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=52 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=199 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=245 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=52 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=199 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=245 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center></div><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"></table><br><p><font size=3> </font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Philipp Grau <phgrau@zedat.fu-berlin.de></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">13/03/2020 13:49</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">[EXTERNAL] [gpfsug-discuss]
maxStatCache and maxFilesToCache: Tip "gpfs_maxstatcache_low".</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>Hello,<br><br>we have a two node NSD cluster based on a DDN system. Currently we<br>run Spectrum Scale 5.0.4.1 in an HPC environment.<br><br>Mmhealth shows a tip stating "gpfs_maxstatcache_low". Our current
settings are:<br><br># mmdiag --config | grep -i cache<br> ! maxFilesToCache 3000000<br> maxStatCache 10000<br><br>maxFilesToCache was tuned during installion and maxStatCache is the<br>according default value.<br><br>After discussing this issue on the german spectumscale meeting, I<br>understand that it is difficult to give a formula on howto calulate<br>this values.<br><br>But I learnt that a FilesToCache entry costs about 10 kbytes of memory<br>and a StatCache entry about 500 bytes. And typically maxStatCache<br>should (obviously) be greater than maxFilesToCache. There is a average<br>100 GB memory usage on our systems (with a total of 265 GB RAM). <br><br>So setting maxStatCache to at least 3000000 should be no problem. But<br>is that correct or to high/low?<br><br>Has anyone some hints or thoughts on this topic? Help is welcome.<br><br>Regards,<br><br>Philipp<br><br>-- <br> Philipp Grau | Freie
Universitaet Berlin <br> phgrau@ZEDAT.FU-Berlin.DE | Zentraleinrichtung fuer Datenverarbeitung<br> Tel: +49 (30) 838 56583 | Fabeckstr. 32 <br> Fax: +49 (30) 838 56721 | 14195 Berlin <br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>