<html><body><p>The Spectrum Scale Knowledge Center does have a topic on collecting CES log files. This might be helpful (4.2.2 version): <a href="http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.scale.v4r22.doc/bl1pdg_ces_monitor_admin.htm"></a><br><a href="http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.scale.v4r22.doc/bl1pdg_ces_monitor_admin.htm"></a><br><a href="http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.scale.v4r22.doc/bl1pdg_ces_monitor_admin.htm">http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.scale.v4r22.doc/bl1pdg_ces_monitor_admin.htm</a><br><br>Alecia A. Ramsay, PMP®<br>Program Manager, New Technology Introduction<br>IBM Systems - Storage<br>aleciarm@us.ibm.com<br>work: 919-435-6494; mobile: 651-260-4928<br><a href="https://www-01.ibm.com/marketing/iwm/iwmdocs/web/cc/earlyprograms/systems.shtml">https://www-01.ibm.com/marketing/iwm/iwmdocs/web/cc/earlyprograms/systems.shtml</a><br><br><img width="16" height="16" src="cid:1__=8FBB0A35DFC24FE98f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for "Sobey, Richard A" ---01/12/2017 04:51:40 AM---Thanks Christof. Would this patch have made it in to C"><font color="#424282">"Sobey, Richard A" ---01/12/2017 04:51:40 AM---Thanks Christof. Would this patch have made it in to CES/GPFS 4.2.1-2.. from what you say probably n</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">"Sobey, Richard A" <r.sobey@imperial.ac.uk></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">01/12/2017 04:51 AM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [gpfsug-discuss] CES log files</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt>Thanks Christof. Would this patch have made it in to CES/GPFS 4.2.1-2.. from what you say probably not?<br><br>This whole incident was caused by a scheduled and extremely rare shutdown of our main datacentre for electrical testing. It's not something that's likely to happen again if at all so reproducing it will be nigh on impossible.<br><br>Food for thought though!<br><br>Richard<br><br>-----Original Message-----<br>From: gpfsug-discuss-bounces@spectrumscale.org [</tt><tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a></tt><tt>] On Behalf Of Christof Schmitt<br>Sent: 11 January 2017 22:33<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Subject: Re: [gpfsug-discuss] CES log files<br><br>A winbindd process taking up 100% could be caused by the problem documented in </tt><tt><a href="https://bugzilla.samba.org/show_bug.cgi?id=12105">https://bugzilla.samba.org/show_bug.cgi?id=12105</a></tt><tt><br><br>Capturing a brief strace of the affected process and reporting that through a PMR would be helpful to debug this problem and provide a fix.<br><br>To answer the wider question: Log files are kept in /var/adm/ras/. In case more detailed traces are required, use the mmprotocoltrace command.<br><br>Regards,<br><br>Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ<br>christof.schmitt@us.ibm.com || +1-520-799-2469 (T/L: 321-2469)<br><br><br><br>From: "Sobey, Richard A" <r.sobey@imperial.ac.uk><br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Date: 01/11/2017 07:00 AM<br>Subject: Re: [gpfsug-discuss] CES log files<br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br><br><br><br>Thanks. Some of the node would just say “failed” or “degraded” with the DCs offline. Of those that thought they were happy to host a CES IP address, they did not respond and winbindd process would take up 100% CPU as seen through top with no users on it.<br> <br>Interesting that even though all CES nodes had the same configuration, three of them never had a problem at all.<br> <br>JF – I’ll look at the protocol tracing next time this happens. It’s a rare thing that three DCs go offline at once but even so there should have been enough resiliency to cope.<br> <br>Thanks<br>Richard<br> <br>From: gpfsug-discuss-bounces@spectrumscale.org [ </tt><tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a></tt><tt>] On Behalf Of Andrew Beattie<br>Sent: 11 January 2017 09:55<br>To: gpfsug-discuss@spectrumscale.org<br>Cc: gpfsug-discuss@spectrumscale.org<br>Subject: Re: [gpfsug-discuss] CES log files<br> <br>mmhealth might be a good place to start<br> <br>CES should probably throw a message along the lines of the following:<br> <br>mmhealth shows something is wrong with AD server:<br>...<br>CES DEGRADED ads_down <br>...<br>Andrew Beattie<br>Software Defined Storage - IT Specialist<br>Phone: 614-2133-7927<br>E-mail: abeattie@au1.ibm.com<br> <br> <br>----- Original message -----<br>From: "Sobey, Richard A" <r.sobey@imperial.ac.uk> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: "'gpfsug-discuss@spectrumscale.org'" <gpfsug-discuss@spectrumscale.org<br>><br>Cc:<br>Subject: [gpfsug-discuss] CES log files<br>Date: Wed, Jan 11, 2017 7:27 PM<br> <br>Which files do I need to look in to determine what’s happening with CES… supposing for example a load of domain controllers were shut down and CES had no clue how to handle this and stopped working until the DCs were switched back on again.<br> <br>Mmfs.log.latest said everything was fine btw.<br> <br>Thanks<br>Richard<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br> <br> _______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br><br><br><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br></tt><br><br><BR>
</body></html>