<html><head></head><body><div class="ydpd790df7cyahoo-style-wrap" style="font-family: Helvetica Neue, Helvetica, Arial, sans-serif; font-size: 13px;"><div></div>
<div>At a guess with no data .... if the application is opening more files than can fit in the maxFilesToCache (MFTC) objects GPFS will expand the MFTC to support the open files, but it will also scan to try and free any unused objects. If you can identify the user job that is causing this you could monitor a system more closely. <br></div><div><br></div><div>Jim<br></div><div><br></div>
</div><div id="yahoo_quoted_3161782924" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Wednesday, November 21, 2018, 2:10:45 AM EST, Saula, Oluwasijibomi <oluwasijibomi.saula@ndsu.edu> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="yiv4325073500">
<style type="text/css"><!--#yiv4325073500 P {margin-top:0;margin-bottom:0;}--></style>
<div dir="ltr">
<div id="yiv4325073500divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri, Helvetica, sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0;">Hello Scalers,</p>
<p style="margin-top:0;margin-bottom:0;"><br>
</p>
<p style="margin-top:0;margin-bottom:0;">First, let me say Happy Thanksgiving to those of us in the US and to those beyond, well, it's a still happy day seeing we're still above ground!
<span>😊</span></p>
<p style="margin-top:0;margin-bottom:0;"><br>
</p>
<p style="margin-top:0;margin-bottom:0;">Now, what I have to discuss isn't anything extreme so don't skip the turkey for this, but lately, on a few of our compute GPFS client nodes, we've been noticing high CPU usage by the mmfsd process and are wondering why.
Here's a sample:</p>
<p style="margin-top:0;margin-bottom:0;"><br>
</p>
<p style="margin-top:0;margin-bottom:0;"></p>
<p style="margin:0px;font:16px Menlo;color:rgb(0, 0, 0);background-color:rgb(254, 244, 156);">
<span style="">[~]# top -b -n 1 | grep mmfs</span></p>
<p style="margin:0px;font:16px Menlo;color:rgb(0, 0, 0);background-color:rgb(254, 244, 156);">
<span style=""><span class="yiv4325073500Apple-converted-space"> </span>PID USER<span class="yiv4325073500Apple-converted-space">
</span>PR<span class="yiv4325073500Apple-converted-space"> </span>NI<span class="yiv4325073500Apple-converted-space">
</span>VIRT<span class="yiv4325073500Apple-converted-space"> </span>RES<span class="yiv4325073500Apple-converted-space">
</span>SHR S<span class="yiv4325073500Apple-converted-space"> </span>%CPU %MEM <span class="yiv4325073500Apple-converted-space">
</span>TIME+ COMMAND</span></p>
<p></p>
<p style="margin:0px;font:16px Menlo;color:rgb(0, 0, 0);background-color:rgb(254, 244, 156);">
<span style=""><br>
</span></p>
<p></p>
<p style="margin-top:0;margin-bottom:0;"></p>
<p style="margin:0px;font:16px Menlo;color:rgb(0, 0, 0);background-color:rgb(254, 244, 156);">
<span style="">231898 root <span class="yiv4325073500Apple-converted-space">
</span>0 -20 14.508g 4.272g<span class="yiv4325073500Apple-converted-space"> </span>70168 S<span class="yiv4325073500Apple-converted-space">
</span>93.8<span class="yiv4325073500Apple-converted-space"> </span>6.8<span class="yiv4325073500Apple-converted-space">
</span>69503:41 </span><span style="color:rgb(189, 49, 27);"><b>mmfs</b></span><span style="">d</span></p>
<p style="margin:0px;font:16px Menlo;color:rgb(0, 0, 0);background-color:rgb(254, 244, 156);">
<span style=""><span class="yiv4325073500Apple-converted-space">
</span>4161 root <span class="yiv4325073500Apple-converted-space"> </span>0 -20<span class="yiv4325073500Apple-converted-space">
</span>121876 <span class="yiv4325073500Apple-converted-space"> </span>9412 <span class="yiv4325073500Apple-converted-space">
</span>1492 S <span class="yiv4325073500Apple-converted-space"> </span>0.0<span class="yiv4325073500Apple-converted-space">
</span>0.0 <span class="yiv4325073500Apple-converted-space"> </span>0:00.22 run</span><span style="color:rgb(189, 49, 27);"><b>mmfs</b></span></p>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0;">Obviously, this behavior was likely triggered by a not-so-convenient user job that in most cases is long finished by the time we investigate. Nevertheless, does anyone have an idea why this might be happening? Any thoughts
on preventive steps even?</p>
<p style="margin-top:0;margin-bottom:0;"><br>
</p>
<p style="margin-top:0;margin-bottom:0;">This is GPFS v4.2.3 on Redhat 7.4, btw...</p>
<p style="margin-top:0;margin-bottom:0;"><br>
</p>
<div id="yiv4325073500Signature">
<div id="yiv4325073500divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0, 0, 0);font-family:Calibri, Arial, Helvetica, sans-serif, Helvetica, EmojiFont, Color UI NotoColorEmoji, UI EmojiSymbols;">
<p></p>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<font face="Calibri,Arial,Helvetica,sans-serif">Thanks,</font></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<font face="Calibri,Arial,Helvetica,sans-serif"> </font></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<font face="Calibri,Arial,Helvetica,sans-serif">Siji Saula</font></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<font face="Calibri,Arial,Helvetica,sans-serif">HPC System Administrator</font></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<font face="Calibri,Arial,Helvetica,sans-serif">Center for Computationally Assisted Science & Technology</font></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<b>NORTH DAKOTA STATE UNIVERSITY</b></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
</div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<br>
</div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<a rel="nofollow" target="_blank" href="https://www.ndsu.edu/alphaindex/buildings/Building::395" id="yiv4325073500LPNoLP"><span id="yiv4325073500LPNoLP"></span></a><span id="yiv4325073500LPNoLP" style="font-family:Calibri, Arial, Helvetica, sans-serif, serif, EmojiFont;font-size:16px;"><a rel="nofollow" target="_blank" href="https://www.ndsu.edu/alphaindex/buildings/Building::396" id="yiv4325073500LPNoLP"><span style="color:rgb(0,111,201);">Research
2 Building</span></a><a rel="nofollow" target="_blank" href="https://www.ndsu.edu/alphaindex/buildings/Building::395" id="yiv4325073500LPNoLP"></a></span><span style="font-family:Calibri, Arial, Helvetica, sans-serif, serif, EmojiFont;font-size:16px;"> – </span><span class="yiv4325073500contextualExtensionHighlight yiv4325073500ms-font-color-themePrimary yiv4325073500ms-border-color-themePrimary yiv4325073500ident_1641_1697" style="font-family:Calibri, Arial, Helvetica, sans-serif, serif, EmojiFont;font-size:16px;color:rgb(0,0,0);">Room
220B</span></div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
Dept 4100, PO Box 6050 / Fargo, ND 58108-6050</div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
p:701.231.7749</div>
<div style="font-family:Calibri, Arial, Helvetica, sans-serif;font-size:16px;margin-top:0px;margin-bottom:0px;">
<a rel="nofollow" id="yiv4325073500LPNoLP"><span id="yiv4325073500LPNoLP">www.ccast.ndsu.edu</span></a> | <a rel="nofollow" id="yiv4325073500LPNoLP"><span id="yiv4325073500LPNoLP">www.ndsu.edu</span></a></div>
<br>
<p></p>
</div>
</div>
</div>
</div>
</div>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br></div>
</div>
</div></body></html>