<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I have opened a PMR, and the official response reflects what you just posted.<div class="">In addition, it seems there are some performance issues with Python 2 that will be </div><div class="">improved with eventual migration to Python 3. I was unaware of the mmhealth</div><div class="">functions that the mmsysmon daemon provides. The impact we were seeing </div><div class="">was some variation in MPI benchmark results when the nodes were fully loaded.</div><div class="">I suppose it would be possible to turn off mmsysmon during the benchmarking,</div><div class="">but I appreciate the effort at streamlining the monitor service. Cutting back on</div><div class="">fork/exec, better python, less polling, more notifications… all good.</div><div class=""><br class=""></div><div class="">Thanks for the details,</div><div class=""><br class=""></div><div class=""> — ddj</div><div class=""><br class=""></div><div class=""><div><blockquote type="cite" class=""><div class="">On Jul 19, 2017, at 9:05 AM, Mathias Dietz <<a href="mailto:MDIETZ@de.ibm.com" class="">MDIETZ@de.ibm.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><font size="2" face="sans-serif" class="">thanks for the feedback. </font><br class=""><br class=""><font size="2" face="sans-serif" class="">Let me clarify what mmsysmon is doing.</font><br class=""><font size="2" face="sans-serif" class="">Since IBM Spectrum Scale 4.2.1 the mmsysmon
process is used for the overall health monitoring and CES failover handling.</font><br class=""><font size="2" face="sans-serif" class="">Even without CES it is an essential
part of the system because it monitors the individual components and provides
health state information and error events. </font><br class=""><font size="2" face="sans-serif" class="">This information is needed by other
Spectrum Scale components (mmhealth command, the IBM Spectrum Scale GUI,
Support tools, Install Toolkit,..) and therefore disabling mmsysmon will
impact them. </font><br class=""><br class=""><tt class=""><font size="2" class="">> It’s a huge problem. I don’t understand why
it hasn’t been given <br class="">> much credit by dev or support.</font></tt><br class=""><br class=""><font size="2" face="sans-serif" class="">Over the last couple of month, the development
team has put a strong focus on this topic. </font><br class=""><font size="2" face="sans-serif" class="">In order to monitor the health of the
individual components, mmsysmon listens for notifications/callback but
also has to do some polling.</font><br class=""><font size="2" face="sans-serif" class="">We are trying to reduce the polling
overhead constantly and replace polling with notifications when possible.
</font><br class=""><br class=""><font size="2" face="sans-serif" class="">Several improvements have been added
to 4.2.3, including the ability to configure the polling frequency to reduce
the overhead. (<i class="">mmhealth config interval) </i></font><br class=""><font size="2" face="sans-serif" class="">See </font><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmhealth.htm" class=""><font size="2" color="blue" face="sans-serif" class="">https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmhealth.htm</font></a><br class=""><font size="2" face="sans-serif" class="">In addition a new option has been introduced
to clock align the monitoring threads in order to reduce CPU jitter. </font><br class=""><br class=""><font size="2" face="sans-serif" class="">Nevertheless, we don't see significant
CPU consumption by mmsysmon on our test systems. </font><br class=""><font size="2" face="sans-serif" class="">It might be a problem specific to your
system environment or a wrong configuration therefore please get in contact
with IBM support to analyze the root cause of the high usage.</font><br class=""><br class=""><font size="2" face="sans-serif" class="">Kind regards<br class=""><br class="">Mathias Dietz<br class=""></font><br class=""><font size="2" face="sans-serif" class="">IBM Spectrum Scale - Release Lead Architect
and RAS Architect <br class=""></font><br class=""><br class=""><tt class=""><font size="2" class=""><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a> wrote on
07/18/2017 07:51:21 PM:<br class=""><br class="">> From: Jonathon A Anderson <<a href="mailto:jonathon.anderson@colorado.edu" class="">jonathon.anderson@colorado.edu</a>></font></tt><br class=""><tt class=""><font size="2" class="">> To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="">gpfsug-discuss@spectrumscale.org</a>></font></tt><br class=""><tt class=""><font size="2" class="">> Date: 07/18/2017 07:51 PM</font></tt><br class=""><tt class=""><font size="2" class="">> Subject: Re: [gpfsug-discuss] mmsysmon.py revisited</font></tt><br class=""><tt class=""><font size="2" class="">> Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a></font></tt><br class=""><tt class=""><font size="2" class="">> <br class="">> There’s no official way to cleanly disable it so far as I know yet;
<br class="">> but you can defacto disable it by deleting /var/mmfs/mmsysmon/<br class="">> mmsysmonitor.conf.<br class="">> <br class="">> It’s a huge problem. I don’t understand why it hasn’t been given
<br class="">> much credit by dev or support.<br class="">> <br class="">> ~jonathon<br class="">> <br class="">> <br class="">> On 7/18/17, 11:21 AM, "<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a>
on <br class="">> behalf of David Johnson" <<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a>
<br class="">> on behalf of <a href="mailto:david_johnson@brown.edu" class="">david_johnson@brown.edu</a>> wrote:<br class="">> <br class="">> <br class="">> <br class="">> <br class="">> We also noticed a fair amount of CPU time accumulated
by mmsysmon.py on<br class="">> our diskless compute nodes. I read the earlier query,
where it <br class="">> was answered:<br class="">> <br class="">> <br class="">> <br class="">> <br class="">> ces == Cluster Export Services, mmsysmon.py comes
from <br class="">> mmcesmon. It is used for managing export services of GPFS. If it is
<br class="">> killed, your nfs/smb etc will be out of work.<br class="">> Their overhead is small and they are very important.
Don't <br class="">> attempt to kill them.<br class="">> <br class="">> <br class="">> <br class="">> <br class="">> <br class="">> <br class="">> Our question is this — we don’t run the latest “protocols",
our <br class="">> NFS is CNFS, and our CIFS is clustered CIFS.<br class="">> I can understand it might be needed with Ganesha, but
on every node? <br class="">> <br class="">> <br class="">> Why in the world would I be getting this daemon running
on all <br class="">> client nodes, when I didn’t install the “protocols" version
<br class="">> of the distribution? We have release 4.2.2 at
the moment. How<br class="">> can we disable this?<br class="">> <br class="">> <br class="">> Thanks,<br class="">> — ddj<br class="">> <br class="">> <br class="">> _______________________________________________<br class="">> gpfsug-discuss mailing list<br class="">> gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class=""><tt class=""><font size="2" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt class=""><font size="2" class=""><br class=""></font></tt><br class="">_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class=""></div></blockquote></div><br class=""></div></body></html>