<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Yeah that did it, it was set to the default value of “no”.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">What exactly does “no” mean as opposed to “yes”? The docs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adm_tuningguide.htm">https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adm_tuningguide.htm</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Aren’t very forthcoming on this …<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(note it looks like we also have to set this in multi-cluster environments in client clusters as well)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Simon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">"Robert.Oesterlin@nuance.com" <Robert.Oesterlin@nuance.com><br>
<b>Date: </b>Friday, 13 April 2018 at 21:17<br>
<b>To: </b>"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
<b>Cc: </b>"Simon Thompson (IT Research Support)" <S.J.Thompson@bham.ac.uk><br>
<b>Subject: </b>Re: [Replicated and non replicated data<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><a name="_MailOriginalBody">Add:<o:p></o:p></a></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">unmountOnDiskFail=meta<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">To your config. You can add it with “-I” to have it take effect w/o reboot.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">Bob Oesterlin<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">Sr Principal Storage Engineer, Nuance<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><b><span style="color:black">From:
</span></b><span style="color:black"><gpfsug-discuss-bounces@spectrumscale.org> on behalf of "Simon Thompson (IT Research Support)" <S.J.Thompson@bham.ac.uk><br>
<b>Reply-To: </b>gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Date: </b>Friday, April 13, 2018 at 3:06 PM<br>
<b>To: </b>"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
<b>Subject: </b>[EXTERNAL] [gpfsug-discuss] Replicated and non replicated data</span><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">I have a question about file-systems with replicated an non replicated data.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">We have a file-system where metadata is set to copies=2 and data copies=2, we then use a placement policy to selectively replicate some data only once based on
file-set. We also place the non-replicated data into a specific pool (6tnlsas) to ensure we know where it is placed.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">My understanding was that in doing this, if we took the disks with the non replicated data offline, we’d still have the FS available for users as the metadata is
replicated. Sure accessing a non-replicated data file would give an IO error, but the rest of the FS should be up.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">We had a situation today where we wanted to take stg01 offline today, so tried using mmchdisk stop -d …. Once we got to about disk stg01-01_12_12, GPFS would refuse
to stop any more disks and complain about too many disks, similarly if we shutdown the NSD servers hosting the disks, the filesystem would have an SGPanic and force unmount.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">First, am I correct in thinking that a FS with non-replicated data, but replicated metadata should still be accessible (not the non-replicated data) when the LUNS
hosting it are down?</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">If so, any suggestions why my FS is panic-ing when we take down the one set of disks?</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">I thought at first we had some non-replicated metadata, tried a mmrestripefs -R –metadata-only to force it to ensure 2 replicas, but this didn’t help.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">Running 5.0.0.2 on the NSD server nodes.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">(First time we went round this we didn’t have a FS descriptor disk, but you can see below that we added this)</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">Thanks</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt">Simon</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:11.0pt"> </span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"> <o:p></o:p></span></p>
</div>
</body>
</html>