<html><head></head><body><div style="color:#000; background-color:#fff; font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px"><div id="yiv6598521023"><div id="yui_3_16_0_ym19_1_1500900502762_16662"><div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;" id="yui_3_16_0_ym19_1_1500900502762_16661"><div id="yiv6598521023yui_3_16_0_ym19_1_1500900502762_12131">There are 3 places that the GPFS mmfsd uses memory the pagepool plus 2 shared memory segments. To see the memory utilization of the shared memory segments run the command mmfsadm dump malloc . The statistics for memory pool id 2 is where maxFilesToCache/maxStatCache objects are and the manager nodes use memory pool id 3 to track the MFTC/MSC objects. <br clear="none"></div><div id="yiv6598521023yui_3_16_0_ym19_1_1500900502762_13732"><br clear="none"></div><div id="yiv6598521023yui_3_16_0_ym19_1_1500900502762_13733">You might want to upgrade to later PTF as there was a PTF to fix a memory leak that occurred in tscomm associated with network connection drops. <br clear="none"></div><div id="yiv6598521023yui_3_16_0_ym19_1_1500900502762_7195"><span></span></div> <div class="yiv6598521023qtdSeparateBR" id="yui_3_16_0_ym19_1_1500900502762_16707"><br clear="none"><br clear="none"></div><div class="yiv6598521023yqt2983717346" id="yiv6598521023yqt44105"></div></div></div></div><div class=".yiv6598521023yahoo_quoted"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"> <div dir="ltr"><font size="2" face="Arial"> On Monday, July 24, 2017 5:29 AM, Peter Childs <p.childs@qmul.ac.uk> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> <div class="yiv6598521023y_msg_container"><div dir="ltr">We have two GPFS clusters.<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">One is fairly old and running 4.2.1-2 and non CCR and the nodes run<br clear="none"></div><div dir="ltr">fine using up about 1.5G of memory and is consistent (GPFS pagepool is<br clear="none"></div><div dir="ltr">set to 1G, so that looks about right.)<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">The other one is "newer" running 4.2.1-3 with CCR and the nodes keep<br clear="none"></div><div dir="ltr">increasing in there memory usage, starting at about 1.1G and are find<br clear="none"></div><div dir="ltr">for a few days however after a while they grow to 4.2G which when the<br clear="none"></div><div dir="ltr">node need to run real work, means the work can't be done.<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">I'm losing track of what maybe different other than CCR, and I'm trying<br clear="none"></div><div dir="ltr">to find some more ideas of where to look.<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">I'm checked all the standard things like pagepool and maxFilesToCache<br clear="none"></div><div dir="ltr">(set to the default of 4000), workerThreads is set to 128 on the new<br clear="none"></div><div dir="ltr">gpfs cluster (against default 48 on the old) <br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">I'm not sure what else to look at on this one hence why I'm asking the<br clear="none"></div><div dir="ltr">community.<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">Thanks in advance<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">Peter Childs<br clear="none"></div><div dir="ltr">ITS Research Storage<br clear="none"></div><div dir="ltr">Queen Mary University of London.<br clear="none"></div><div dir="ltr">_______________________________________________<br clear="none"></div><div dir="ltr">gpfsug-discuss mailing list<br clear="none"></div><div dir="ltr">gpfsug-discuss at spectrumscale.org<br clear="none"></div><div dir="ltr"><a rel="nofollow" shape="rect" target="_blank" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br clear="none"></div><br clear="none"><br clear="none"></div> </div> </div> </div></div></body></html>