<html><body><p><font size="2">To get an explanation for any event one can ask the system:<br></font><tt><br></tt><tt><font size="2"># </font></tt><tt><b><font size="2">mmhealth event show fserrinvalid </font></b></tt><tt><font size="2"> <br>Event Name: fserrinvalid <br>Event ID: 999338 <br>Description: Unrecognized FSSTRUCT error received. Check documentation <br>Cause: A filesystem corruption detected <br>User Action: Check error message for details and the mmfs.log.latest log for further details. See the topic Checking and repairing a file system in the IBM Spectrum Scale documentation: Administering. Managing file systems. If the file system is severely damaged, the best course of action is to follow the procedures in section: Additional information to collect for file system corruption or MMFS_FSSTRUCT errors <br>Severity: ERROR <br>State: DEGRADED </font></tt><tt><br></tt><br><font size="2">The event is triggered by a callback which may not fire on all nodes, that is why only a subset of nodes have the information.<br>Depending on the version of scale the procedure to remove the event varies: For newer release please use<br><br></font><tt><font size="2"># mmhealth event resolve</font></tt><br><tt><font size="2">Missing arguments.</font></tt><br><tt><font size="2">Usage:</font></tt><br><tt><font size="2"> mmhealth event resolve {EventName} [Identifier]</font></tt><br><br><font size="2">For older releases it is described here: </font><a href="https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.5/com.ibm.spectrum.scale.v5r05.doc/bl1pdg_fsstruc.htm"><font size="2">https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.5/com.ibm.spectrum.scale.v5r05.doc/bl1pdg_fsstruc.htm</font></a><br><tt>mmsysmonc event filesystem fsstruct_fixed <filesystem_name> <filesystem_name></tt><br><br><br><font size="2" face="Arial">Mit freundlichen Grüßen / Kind regards</font><br><br><b><font face="Arial">Norbert Schuld</font></b><br><br><font size="1" color="#0055AA" face="Arial">M925:IBM Spectrum Scale Software Development</font><table border="0" cellspacing="0" cellpadding="0"><tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#696969; "></td></tr>
<tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><img width="1" height="1" src="cid:1__=4EBB0CB9DFC889528f9e8a93df938690918c4EB@" border="0" alt=""></td></tr>
<tr valign="top"><td class="inner" width="76" valign="middle"><font size="1" color="#0055AA" face="Arial">Phone:</font></td><td class="inner" width="275" valign="middle"><font size="1" color="#0055AA" face="Arial">+49-160 70 70 335</font></td><td class="inner" width="330" valign="middle"><font size="1" color="#0055AA" face="Arial"> IBM Deutschland Research & Development GmbH</font></td><td width="120" rowspan="3" valign="middle"><div align="right"><img src="cid:2__=4EBB0CB9DFC889528f9e8a93df938690918c4EB@" width="87" height="30" align="bottom"></div></td></tr>
<tr valign="top"><td class="inner" width="76" valign="middle"><font size="1" color="#0055AA" face="Arial">Email:</font></td><td class="inner" width="275" valign="middle"><font size="1" color="#0055AA" face="Arial">nschuld@de.ibm.com</font></td><td class="inner" width="330" valign="middle"><font size="1" color="#0055AA" face="Arial"> Am Weiher 24</font></td></tr>
<tr valign="top"><td class="inner" width="76" valign="middle"><img width="1" height="1" src="cid:1__=4EBB0CB9DFC889528f9e8a93df938690918c4EB@" border="0" alt=""></td><td class="inner" width="275" valign="middle"><img width="1" height="1" src="cid:1__=4EBB0CB9DFC889528f9e8a93df938690918c4EB@" border="0" alt=""></td><td class="inner" width="330" valign="middle"><font size="1" color="#0055AA" face="Arial"> 65451 Kelsterbach</font></td></tr>
<tr valign="top"><td class="inner2" width="800" colspan="4" valign="middle"><i><font size="1" face="Arial">Knowing is not enough; we must apply. Willing is not enough; we must do.</font></i></td></tr>
<tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#696969; "></td></tr>
<tr valign="top"><td class="foot" width="800" colspan="4" valign="middle"><a href="https://www.ibm.com/privacy/us/en/"><u><font size="1" color="#0000FF" face="Arial">IBM Data Privacy Statement</font></u></a><font size="1" color="#999999" face="Arial"> </font></td></tr>
<tr valign="top"><td class="foot" width="800" colspan="4" valign="middle"><font size="1" color="#999999" face="Arial">IBM Deutschland Research & Development GmbH / Vorsitzender des Aufsichtsrats: Gregor Pillen<br>Geschäftsführung: Dirk Wittkopp<br>Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294 </font></td></tr></table><br><br><img width="16" height="16" src="cid:3__=4EBB0CB9DFC889528f9e8a93df938690918c4EB@" border="0" alt="Inactive hide details for Prasad Surampudi ---24.11.2020 17:05:31---We are seeing fserrinvalid error on couple of filesystems i"><font size="2" color="#424282">Prasad Surampudi ---24.11.2020 17:05:31---We are seeing fserrinvalid error on couple of filesystems in Spectrum Scale cluster. These errors ar</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Prasad Surampudi <prasad.surampudi@theatsgroup.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">24.11.2020 17:05</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">[EXTERNAL] [gpfsug-discuss] mmhealth reports fserrinvalid errors on CNFS servers</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><font face="Calibri">We are seeing fserrinvalid error on couple of filesystems in Spectrum Scale cluster. These errors are reported but mmhealth only couple of nodes (CNFS servers) in the cluster, but mmhealth on other nodes shows no issues. Any idea what this error means? And why its reported on CNFS servers and not on other nodes? What need to be done to fix this issue?</font><br><font face="Calibri"> </font><br><font face="Calibri">sudo /usr/lpp/mmfs/bin/mmhealth node show FILESYSTEM -v</font><br><font face="Calibri"> </font><br><font face="Calibri">Node name: cnfs05-gpfs</font><br><font face="Calibri"> </font><br><font face="Calibri">Component Status Reasons</font><br><font face="Calibri">-------------------------------------------------------------------</font><br><font face="Calibri">FILESYSTEM DEGRADED fserrinvalid(vol)</font><br><font face="Calibri"> argus HEALTHY -</font><br><font face="Calibri"> dytech HEALTHY -</font><br><font face="Calibri"> enlnt_E HEALTHY -</font><br><font face="Calibri"> enlnt_Es HEALTHY -</font><br><font face="Calibri"> haaforfs HEALTHY -</font><br><font face="Calibri"> haaforfs2 HEALTHY -</font><br><font face="Calibri"> historical HEALTHY -</font><br><font face="Calibri"> prcfs HEALTHY -</font><br><font face="Calibri"> qmtfs HEALTHY -</font><br><font face="Calibri"> research HEALTHY -</font><br><font face="Calibri"> research2 HEALTHY -</font><br><font face="Calibri"> schon_raw HEALTHY -</font><br><font face="Calibri"> uhdb_vol1 HEALTHY -</font><br><font face="Calibri"> vol DEGRADED fserrinvalid(vol)</font><br><font face="Calibri"> </font><br><font face="Calibri">Event Parameter Severity Event Message</font><br><font face="Calibri">----------------------------------------------------------------------------------------------------------</font><br><font face="Calibri">fserrinvalid vol ERROR FS=vol,ErrNo=1124,Unknown error=0464000000010000000180A108BC000079B4000000000000003400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000</font><tt><font size="2">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></tt><tt><font size="2"> <br></font></tt><br><br><BR>
</body></html>