[gpfsug-discuss] mmhealth reports fserrinvalid errors on CNFS servers
Norbert Schuld
NSCHULD at de.ibm.com
Tue Nov 24 16:44:35 GMT 2020
To get an explanation for any event one can ask the system:
# mmhealth event show fserrinvalid
Event Name: fserrinvalid
Event ID: 999338
Description: Unrecognized FSSTRUCT error received. Check
documentation
Cause: A filesystem corruption detected
User Action: Check error message for details and the
mmfs.log.latest log for further details. See the topic Checking and
repairing a file system in the IBM Spectrum Scale documentation:
Administering. Managing file systems. If the file system is severely
damaged, the best course of action is to follow the procedures in section:
Additional information to collect for file system corruption or
MMFS_FSSTRUCT errors
Severity: ERROR
State: DEGRADED
The event is triggered by a callback which may not fire on all nodes, that
is why only a subset of nodes have the information.
Depending on the version of scale the procedure to remove the event varies:
For newer release please use
# mmhealth event resolve
Missing arguments.
Usage:
mmhealth event resolve {EventName} [Identifier]
For older releases it is described here:
https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.5/com.ibm.spectrum.scale.v5r05.doc/bl1pdg_fsstruc.htm
mmsysmonc event filesystem fsstruct_fixed <filesystem_name>
<filesystem_name>
Mit freundlichen Grüßen / Kind regards
Norbert Schuld
M925:IBM Spectrum Scale Software Development
Phone: +49-160 70 70 335 IBM Deutschland Research & Development
GmbH
Email: nschuld at de.ibm.com Am Weiher 24
65451 Kelsterbach
Knowing is not
enough; we must
apply. Willing is
not enough; we
must do.
IBM Data Privacy
Statement
IBM Deutschland
Research &
Development
GmbH /
Vorsitzender des
Aufsichtsrats:
Gregor Pillen
Geschäftsführung:
Dirk Wittkopp
Sitz der
Gesellschaft:
Böblingen /
Registergericht:
Amtsgericht
Stuttgart, HRB
243294
From: Prasad Surampudi <prasad.surampudi at theatsgroup.com>
To: "gpfsug-discuss at spectrumscale.org"
<gpfsug-discuss at spectrumscale.org>
Date: 24.11.2020 17:05
Subject: [EXTERNAL] [gpfsug-discuss] mmhealth reports fserrinvalid
errors on CNFS servers
Sent by: gpfsug-discuss-bounces at spectrumscale.org
We are seeing fserrinvalid error on couple of filesystems in Spectrum Scale
cluster. These errors are reported but mmhealth only couple of nodes
(CNFS servers) in the cluster, but mmhealth on other nodes shows no issues.
Any idea what this error means? And why its reported on CNFS servers and
not on other nodes? What need to be done to fix this issue?
sudo /usr/lpp/mmfs/bin/mmhealth node show FILESYSTEM -v
Node name: cnfs05-gpfs
Component Status Reasons
-------------------------------------------------------------------
FILESYSTEM DEGRADED fserrinvalid(vol)
argus HEALTHY -
dytech HEALTHY -
enlnt_E HEALTHY -
enlnt_Es HEALTHY -
haaforfs HEALTHY -
haaforfs2 HEALTHY -
historical HEALTHY -
prcfs HEALTHY -
qmtfs HEALTHY -
research HEALTHY -
research2 HEALTHY -
schon_raw HEALTHY -
uhdb_vol1 HEALTHY -
vol DEGRADED fserrinvalid(vol)
Event Parameter Severity Event Message
----------------------------------------------------------------------------------------------------------
fserrinvalid vol ERROR
FS=vol,ErrNo=1124,Unknown
error=0464000000010000000180A108BC000079B4000000000000003400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201124/fff781b9/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201124/fff781b9/attachment-0006.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1D963707.gif
Type: image/gif
Size: 1851 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201124/fff781b9/attachment-0007.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201124/fff781b9/attachment-0008.gif>
More information about the gpfsug-discuss
mailing list