<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I don’t know or care who the hardware vendor is, but they can DEFINITELY ship you a controller with the right firmware! Just demand it, which is what I do and
they have basically always complied with the request. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There is the risk associated with running even longer with a single point of failure, only using the surviving controller, but if this storage system has been
in production a long time (e.g. a year or so) and is generally reliable, then they should be able to get you a new, factory tested controller with the right FW versions in a couple of days.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">The choice is yours of course,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">-Bryan<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> gpfsug-discuss-bounces@spectrumscale.org [mailto:gpfsug-discuss-bounces@spectrumscale.org]
<b>On Behalf Of </b>Steve Xiao<br>
<b>Sent:</b> Thursday, February 08, 2018 11:18 AM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> Re: [gpfsug-discuss] hdisk suspend / stop (Buterbaugh, Kevin L)<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><em><span style="font-size:8.5pt;font-family:"Verdana",sans-serif;color:#333333">Note: External Email</span></em><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">You can change the cluster configuration to online unmount the file system when there is error accessing metadata. This can be done run the following
command:</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif"> mmchconfig unmountOnDiskFail=meta -i
</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">After this configuration change, you should be able to stop all 5 NSDs with mmchdisk stop command. While these NSDs are in down state, any user IO to files resides on these disks will fail but
your file system should state mounted and usable.</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Steve Y. Xiao</span><br>
<span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> Date: Thu, 8 Feb 2018 15:59:44 +0000</tt><br>
<tt>> From: "Buterbaugh, Kevin L" <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu">Kevin.Buterbaugh@Vanderbilt.Edu</a>></tt><br>
<tt>> To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>></tt><br>
<tt>> Subject: [gpfsug-discuss] mmchdisk suspend / stop</tt><br>
<tt>> Message-ID: <<a href="mailto:8DCA682D-9850-4C03-8930-EA6C68B41109@vanderbilt.edu">8DCA682D-9850-4C03-8930-EA6C68B41109@vanderbilt.edu</a>></tt><br>
<tt>> Content-Type: text/plain; charset="utf-8"</tt><br>
<tt>> </tt><br>
<tt>> Hi All,</tt><br>
<tt>> </tt><br>
<tt>> We are in a bit of a difficult situation right now with one of our </tt><br>
<tt>> non-IBM hardware vendors (I know, I know, I KNOW - buy IBM hardware!</tt><br>
<tt>> <grin>) and are looking for some advice on how to deal with this </tt><br>
<tt>> unfortunate situation.</tt><br>
<tt>> </tt><br>
<tt>> We have a non-IBM FC storage array with dual-?redundant? </tt><br>
<tt>> controllers. One of those controllers is dead and the vendor is </tt><br>
<tt>> sending us a replacement. However, the replacement controller will </tt><br>
<tt>> have mis-matched firmware with the surviving controller and - long </tt><br>
<tt>> story short - the vendor says there is no way to resolve that </tt><br>
<tt>> without taking the storage array down for firmware upgrades. </tt><br>
<tt>> Needless to say there?s more to that story than what I?ve included </tt><br>
<tt>> here, but I won?t bore everyone with unnecessary details.</tt><br>
<tt>> </tt><br>
<tt>> The storage array has 5 NSDs on it, but fortunately enough they are </tt><br>
<tt>> part of our ?capacity? pool ? i.e. the only way a file lands here is</tt><br>
<tt>> if an mmapplypolicy scan moved it there because the *access* time is</tt><br>
<tt>> greater than 90 days. Filesystem data replication is set to one.</tt><br>
<tt>> </tt><br>
<tt>> So ? what I was wondering if I could do is to use mmchdisk to either</tt><br>
<tt>> suspend or (preferably) stop those NSDs, do the firmware upgrade, </tt><br>
<tt>> and resume the NSDs? The problem I see is that suspend doesn?t stop</tt><br>
<tt>> I/O, it only prevents the allocation of new blocks ? so, in theory, </tt><br>
<tt>> if a user suddenly decided to start using a file they hadn?t needed </tt><br>
<tt>> for 3 months then I?ve got a problem. Stopping all I/O to the disks</tt><br>
<tt>> is what I really want to do. However, according to the mmchdisk man</tt><br>
<tt>> page stop cannot be used on a filesystem with replication set to one.</tt><br>
<tt>> </tt><br>
<tt>> There?s over 250 TB of data on those 5 NSDs, so restriping off of </tt><br>
<tt>> them or setting replication to two are not options.</tt><br>
<tt>> </tt><br>
<tt>> It is very unlikely that anyone would try to access a file on those </tt><br>
<tt>> NSDs during the hour or so I?d need to do the firmware upgrades, but</tt><br>
<tt>> how would GPFS itself react to those (suspended) disks going away </tt><br>
<tt>> for a while? I?m thinking I could be OK if there was just a way to </tt><br>
<tt>> actually stop them rather than suspend them. Any undocumented </tt><br>
<tt>> options to mmchdisk that I?m not aware of???</tt><br>
<tt>> </tt><br>
<tt>> Are there other options - besides buying IBM hardware - that I am </tt><br>
<tt>> overlooking? Thanks...</tt><br>
<tt>> </tt><br>
<tt>> ?</tt><br>
<tt>> Kevin Buterbaugh - Senior System Administrator</tt><br>
<tt>> Vanderbilt University - Advanced Computing Center for Research and Education</tt><br>
<tt>> Kevin.Buterbaugh@vanderbilt.edu<</tt></span><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu"><tt><span style="font-size:10.0pt">mailto:Kevin.Buterbaugh@vanderbilt.edu</span></tt></a><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> > - (615)875-9633</tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<br>
</span><o:p></o:p></p>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this
email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness
or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial
product.<br>
</font>
</body>
</html>