<font size=2 face="sans-serif">Hi,</font><br><br><font size=2 face="sans-serif">Just remember to take into account the
token memory aspect of the chosen values, especially for large clusters
( check </font><a href="https://www.spectrumscaleug.org/wp-content/uploads/2019/05/SSSD19DE-Day-2-B04-Spectrum-Scale-Memory-Usage.pdf"><font size=3 color=blue><u>https://www.spectrumscaleug.org/wp-content/uploads/2019/05/SSSD19DE-Day-2-B04-Spectrum-Scale-Memory-Usage.pdf</u></font></a><font size=3></font><font size=2 face="sans-serif"> for more details).</font><br><br><font size=2 face="sans-serif"><br>Regards,<br><br>Tomer Perry<br><br></font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Philipp Grau <phgrau@zedat.fu-berlin.de></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">14/03/2020 15:12</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">[EXTERNAL] Re:
[gpfsug-discuss] maxStatCache and maxFilesToCache: Tip"gpfs_maxstatcache_low".</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>Hello all,<br><br>thank you all a lot for the feedback to my question. I think that I now<br>understand the situation better. I will talk with my coworkers and we<br>will find a better setting for the values...<br><br>Many thanks,<br><br>Philipp<br><br>* Felipe Knop <knop@us.ibm.com> [13.03.20 16:22]:<br>> All,<br>> <br>> Looks to me that the demands of the workload will dictate how many
files we<br>> should be cache, that is: maxStatCache + maxFilesToCache .<br>> <br>> The "mix" between maxStatCache and maxFilesToCache depends
on how much memory<br>> can be made available. Accessing files from maxFilesToCache is more
efficient,<br>> but stat cache entries use much less space.<br>> <br>> With the<br>> <br>> ! maxFilesToCache 3000000<br>> maxStatCache 10000<br>> <br>> combination, the stat cache is not providing any significant help,
since only<br>> 0.3% of the files that are cached can fit in the stat cache. If enough
memory<br>> is available then maxStatCache could be increased to (say) 3000000,
at a cost<br>> of 1.4GB. But maxFilesToCache = 3000000 uses up to 27GB. The
next questions<br>> are then<br>> <br>> 1) Can such memory become available on the node, given the pagepool
size ?<br>> <br>> 2) Does the workload require caching that many files?<br>> <br>> <br>> Felipe<br>> <br>> ----<br>> Felipe Knop knop@us.ibm.com<br>> GPFS Development and Security<br>> IBM Systems<br>> IBM Building 008<br>> 2455 South Rd, Poughkeepsie, NY 12601<br>> (845) 433-9314 T/L 293-9314<br>> <br>> <br>> <br>> <br>> ----- Original message -----<br>> From: "Frederick Stock" <stockf@us.ibm.com><br>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>> To: gpfsug-discuss@spectrumscale.org<br>> Cc: gpfsug-discuss@spectrumscale.org<br>> Subject: [EXTERNAL] Re: [gpfsug-discuss] maxStatCache
and maxFilesToCache:<br>> Tip"gpfs_maxstatcache_low".<br>> Date: Fri, Mar 13, 2020 10:01 AM<br>> <br>> As you have learned there is no simple formula for setting
the<br>> maxStatToCache, or for that matter the maxFilesToCache,
configuration<br>> values. Memory is certainly one consideration
but another is directory<br>> listing operations. The information kept in the
stat cache is sufficient<br>> for fulfilling directory listings. If your users
are doing directory<br>> listings regularly then a larger stat cache could be
helpful. <br>> <br>> Fred<br>> __________________________________________________<br>> Fred Stock | IBM Pittsburgh Lab | 720-430-8821<br>> stockf@us.ibm.com<br>> <br>> <br>> <br>> ----- Original message -----<br>> From: Philipp Grau <phgrau@zedat.fu-berlin.de><br>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>> Cc:<br>> Subject: [EXTERNAL] [gpfsug-discuss] maxStatCache
and maxFilesToCache:<br>> Tip "gpfs_maxstatcache_low".<br>> Date: Fri, Mar 13, 2020 8:49 AM<br>> <br>> Hello,<br>> <br>> we have a two node NSD cluster based on
a DDN system. Currently we<br>> run Spectrum Scale 5.0.4.1 in an HPC environment.<br>> <br>> Mmhealth shows a tip stating "gpfs_maxstatcache_low".
Our current<br>> settings are:<br>> <br>> # mmdiag --config | grep -i cache<br>> ! maxFilesToCache 3000000<br>> maxStatCache 10000<br>> <br>> maxFilesToCache was tuned during installion
and maxStatCache is the<br>> according default value.<br>> <br>> After discussing this issue on the german
spectumscale meeting, I<br>> understand that it is difficult to give
a formula on howto calulate<br>> this values.<br>> <br>> But I learnt that a FilesToCache entry
costs about 10 kbytes of memory<br>> and a StatCache entry about 500 bytes.
And typically maxStatCache<br>> should (obviously) be greater than maxFilesToCache.
There is a average<br>> 100 GB memory usage on our systems (with
a total of 265 GB RAM).<br>> <br>> So setting maxStatCache to at least 3000000
should be no problem. But<br>> is that correct or to high/low?<br>> <br>> Has anyone some hints or thoughts on this
topic? Help is welcome.<br>> <br>> Regards,<br>> <br>> Philipp<br>> <br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>