<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css">BODY {
FONT-FAMILY: arial
}
</style>
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal;
font-family:"Arial",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;}
span.E-MailFormatvorlage18
{mso-style-type:personal;
font-family:"Arial",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;}
span.commenttext2
{mso-style-name:commenttext2;}
span.E-MailFormatvorlage20
{mso-style-type:personal-compose;
font-family:"Arial",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="#0563C1" vlink="#954F72">
<p></p>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Hallo All,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">we fixed our Problem here with Spectrum Scale Support. The fixing cmd were ‚</span><span lang="EN" style="font-size:10.0pt;font-family:Consolas">mmcommon recoverfs
tsmconf</span><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">‘ and “</span><span lang="EN" style="font-size:10.0pt;font-family:Consolas">tsdeldisk tsmconf -d "nsd_g4_tsmconf".
</span><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">The final reason<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">for this problem, if I want to delete a disk in a filesystem all disk must be reachable from the requesting host. In our config the NSD-Server had no NSD-Server Definitions
and the <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">Quorum Buster Node had no access to the SAN attached disk.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">A Recommendation from my site here are:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">This should be documented for a high available config with a 3 side implementation, or the cmds that want to update the nsd-descriptors for each disk should check are
any disk reachable and don’t do a SG-Panic.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif">Regards Renar
</span><span lang="EN" style="font-size:12.0pt;font-family:"Arial",sans-serif"> </span><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div></div>
</div>
<p></p>
<p><font style="font-family:Arial;">Renar</font> <font style="font-family:Arial;">Grunenberg</font><br>
<font style="font-family:Arial;">Abteilung Informatik – Betrieb</font><br>
<br>
<font style="font-family:Arial;">HUK-COBURG</font><br>
<font style="font-family:Arial;">Bahnhofsplatz</font><br>
<font style="font-family:Arial;">96444</font> <font style="font-family:Arial;">Coburg</font><br>
<table class="d43f6f84-6537-44ab-9db4-cfa35dc306c7Table" style="font-family:Arial;">
<tbody>
<tr>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">Telefon:</font></font></font></td>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">09561 96-44110</font></font></font></td>
</tr>
<tr>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">Telefax:</font></font></font></td>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">09561 96-44104</font></font></font></td>
</tr>
<tr>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">E-Mail:</font></font></font></td>
<td><font style="font-family:Arial;"><font style="font-family:Arial;">Renar.Grunenberg@huk-coburg.de</font></font></td>
</tr>
<tr>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">Internet:</font></font></font></td>
<td><font style="font-family:Arial;"><font style="font-family:Arial;"><font style="font-family:Arial;">www.huk.de</font></font></font></td>
</tr>
</tbody>
</table>
</p>
<hr id="HR1">
<font face="Arial"><font size="1">HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter Deutschlands a. G. in Coburg<br>
Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021<br>
Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg<br>
Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin.<br>
Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans Olav Herøy, Dr. Jörg Rheinländer (stv.), Sarah Rössler, Daniel Thomas.<br>
</font>
<hr id="HR1">
</font><font size="1" face="Arial">Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen.<br>
Wenn Sie nicht der richtige Adressat sind oder diese Nachricht irrtümlich erhalten haben,<br>
informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht.<br>
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht ist nicht gestattet.<br>
<br>
This information may contain confidential and/or privileged information.<br>
If you are not the intended recipient (or have received this information in error) please notify the<br>
sender immediately and destroy this information.<br>
Any unauthorized copying, disclosure or distribution of the material in this information is strictly forbidden.<br>
</font>
<hr id="HR1">
<p></p>
<p></p>
<p></p>
<div class="WordSection1">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN" style="mso-fareast-language:DE">Von:</span></b><span lang="EN" style="mso-fareast-language:DE"> Grunenberg, Renar
<br>
<b>Gesendet:</b> Mittwoch, 4. </span><span lang="EN-US" style="mso-fareast-language:DE">Juli 2018 07:47<br>
<b>An:</b> 'gpfsug-discuss@spectrumscale.org' <gpfsug-discuss@spectrumscale.org><br>
<b>Betreff:</b> Filesystem Operation error<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">Hallo All,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">follow a short story from yesterday on Version 5.0.1.1. We had a 3 - Node cluster (2 Nodes for IO and the third for a quorum Buster function).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">A Admin make a mistake an take a delete of the 3 Node (VM). We restored ist with a VM Snapshot no Problem. The only point here we lost complete<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">7 desconly disk. We defined new one and want to delete this disk with mmdeldisk. On 6 Filesystems no problem but one has now a Problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">We delete this disk finaly with mmdeldisk fsname -p. And we see now after a successfully mmdelnsd the old disk already in following display.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">mmlsdisk tsmconf -L<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">disk driver sector failure holds holds storage<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">name type size group metadata data status availability disk id pool remarks<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">------------ -------- ------ ----------- -------- ----- ------------- ------------ ------- ------------ ---------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">nsd_tsmconf001_DSK20 nsd 512 0 Yes Yes ready up 1 system desc<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">nsd_g4_tsmconf nsd 512 2 No No removing refs down 2 system<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">nsd_tsmconf001_DSK70 nsd 512 1 Yes Yes ready up 3 system desc<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:Consolas">nsd_g4_tsmconf1 nsd 512 2 No No ready up 4 system desc<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">After that all fs-cmd geneate a fs operation error here like this.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas">Error=MMFS_SYSTEM_UNMOUNT, ID=0xC954F85D, Tag=3882673: Unrecoverable file system operation error. Status code 65536. Volume tsmconf<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">Questions:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">1. What does this mean ‘removing refs’. Now we don’t have the possibility to handle these disk. The disk itself is no more existend, but in the stripegroup a referenz
is available.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas">nsd_g4_tsmconf: uid 0A885085:577BB637, status ReferencesBeingRemoved, availability Unavailable,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> created on node 10.136.80.133, Tue Jul 5 15:29:27 2016<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> type 'nsd', sector size 512, failureConfigVersion 424<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> quorum weight {0,0}, failure group: id 2, fg index 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> locality group: id 2, lg index 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> failureGroupStrP: (2), rackId 2, locationId 0, extLgId 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> nSectors 528384 (0:81000) (258 MB), inode0Sector 131072<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> alloc region: no of bits 0, seg num -1, offset 0, len 72<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> suballocator 0x18015B8A7A4 type 0 nBits 32 subSize 0 dataOffset 4<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> nRows 0 len/off:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> storage pool: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> holds nothing<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> sectors past efficient device boundary: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> isFenced: 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> start Region No: -1 end Region No:-1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas"> start AllocMap Record: -1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">2. Are there any cmd to handle these?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">3. Where can I find the
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas">Status code 65536?</span><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">A PMR is also open.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">Any Hints?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif">Regards Renar<o:p></o:p></span></p>
</div>
<p></p>
<p></p>
<p></p>
<p></p>
<p></p>
<p></p>
<p></p>
<p></p>
</body>
</html>