<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Silly cut/paste error. APAR is
</span><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#1E1F21;background:#F9F8F8">IJ40726<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#1E1F21;background:#F9F8F8"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#1E1F21;background:#F9F8F8">Ed Wahl<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ohio Supercomputer Center<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt">From:</span></b><span style="font-size:11.0pt"> gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org>
<b>On Behalf Of </b>Wahl, Edward<br>
<b>Sent:</b> Thursday, October 13, 2022 12:28 PM<br>
<b>To:</b> gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] Node crashes after upgrading to V5.1.2+ with msgqueue enabled<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">I’ll just toss in here that efixes for this issue are available. As we’re in the tail end of a large data transition and cannot go to LATEST yet, we’re running
some efixes for this and they do work. However mmhealth will still report it broken
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">I’ll just toss in here that efixes for this issue are available. As we’re in the tail end of a large data transition and cannot go to LATEST yet, we’re running some efixes for this and they do work. However
mmhealth will still report it broken until we can delete the msgqueue. On the positive side, the crashes went away.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I don’t recall the 5.1.2.6 efix #s, but 5.1.3.1. is efix26 for ppcle and 5.1.3.1 efix 27 for x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ed Wahl<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ohio Supercomputer Center<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt">From:</span></b><span style="font-size:11.0pt"> gpfsug-discuss <<a href="mailto:gpfsug-discuss-bounces@gpfsug.org">gpfsug-discuss-bounces@gpfsug.org</a>>
<b>On Behalf Of </b>Luke Sudbery<br>
<b>Sent:</b> Thursday, October 13, 2022 6:19 AM<br>
<b>To:</b> <a href="mailto:gpfsug-discuss@gpfsug.org">gpfsug-discuss@gpfsug.org</a><br>
<b>Subject:</b> [gpfsug-discuss] Node crashes after upgrading to V5.1.2+ with msgqueue enabled<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">IBM Spectrum Scale : Node crashes after upgrading to V5. 1. 2+ with msgqueue enabled https: //www. ibm. com/support/pages/node/6824149?myns=s033&mynp=OCSTXKQY&mync=E&cm_sp=s033-_-OCSTXKQY-_-E
says: <o:p>
</o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt">IBM Spectrum Scale : Node crashes after upgrading to V5.1.2+ with msgqueue enabled
<o:p></o:p></span></b></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/www.ibm.com/support/pages/node/6824149?myns=s033&mynp=OCSTXKQY&mync=E&cm_sp=s033-_-OCSTXKQY-_-E__;!!KGKeukY!39sbejpLUCp6er4BjDpjPh0bsTcoitFTC1frKvaULMLFEJk3_YlHVGUCYZ5c9mjAq_QLpUVUgnq5KTrM3Q3Ebw$"><span lang="EN-GB" style="font-size:11.0pt">https://www.ibm.com/support/pages/node/6824149?myns=s033&mynp=OCSTXKQY&mync=E&cm_sp=s033-_-OCSTXKQY-_-E</span></a><span lang="EN-GB" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">says:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;background:#F8F8F8"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB">A node crash can occur due to resource exhaustion after upgrading to <b>V5.1.2.0</b> or
higher with msgqueue enabled. This affects all upgraded nodes until the filesystem version is moved to <b>V5.1.2.0+</b> (<i>mmchfs -V</i>) <b><i>and </i></b>the migration off msgqueue occurs (<i>mmmsgqueue config --remove-msgqueue</i>).<o:p></o:p></span></p>
<p class="MsoNormal" style="background:white"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="background:white"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB">My emphasis on <b><i>and</i></b>. But also says:<o:p></o:p></span></p>
<p class="MsoNormal" style="background:white"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;background:#F8F8F8"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB">This issue will affect all versions mentioned above until <b><i>msgqueue</i> </b>is
disabled across the cluster.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span lang="EN-GB" style="font-size:11.5pt;font-family:"Arial",sans-serif;color:#1D1C1D;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="background:white"><span lang="EN-GB" style="font-size:11.0pt;color:#1D1C1D;mso-fareast-language:EN-GB">So it's not entirely clear whether you have to just update client version, update the filesystem version or disable msgqueue or
do all 3 to fix it...<o:p></o:p></span></p>
<p class="MsoNormal" style="background:white"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Can an IBMer clarify?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Many thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Luke<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">--
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Luke Sudbery<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Principal Engineer (HPC and Storage).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Architecture, Infrastructure and Systems<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Advanced Research Computing, IT Services<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Room 132, Computer Centre G5, Elms Road<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:9.0pt;color:#1F497D;mso-fareast-language:EN-GB">Please note I don’t work on Monday.<o:p></o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>