<div dir="ltr"><div><div>Bryan:<br><br></div>Have you ever attempted to do this knowing that one quorum server is down? *all* nsdservers will not see the nsd about to be added?<br><br></div><div>How about temporarily removing quorum from a nsd server...?<br></div><div><br></div>Thanks<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 5, 2017 at 3:06 PM, Bryan Banister <span dir="ltr"><<a href="mailto:bbanister@jumptrading.com" target="_blank">bbanister@jumptrading.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="#0563C1" vlink="#954F72" lang="EN-US">
<div class="m_-790980191485529841WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">There may be an issue with one of the other NSDs in the file system according to the “</span>mmadddisk: File system home has some disks that are in a non-ready
state.<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">“ message in our output. Best to check the status of the NSDs in the file system using the `mmlsdisk home` and if any disks are not ‘up’ then run the `mmchdisk home start
-a` command after confirming that all nsdservers can see the disks. I typically use `mmdsh -N nsdnodes tspreparedisk –s | dshbak –c` for that.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hope that helps,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">-Bryan<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@<wbr>spectrumscale.org</a> [mailto:<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-<wbr>bounces@spectrumscale.org</a>]
<b>On Behalf Of </b>J. Eric Wonderley<br>
<b>Sent:</b> Thursday, January 05, 2017 2:01 PM<br>
<b>To:</b> gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.<wbr>org</a>><br>
<b>Subject:</b> [gpfsug-discuss] nsd not adding with one quorum node down?<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I have one quorum node down and attempting to add a nsd to a fs:<br>
[root@cl005 ~]# mmadddisk home -F add_1_flh_home -v no |& tee /root/adddisk_flh_home.out<br>
Verifying file system configuration information ...<br>
<br>
The following disks of home will be formatted on node cl003:<br>
r10f1e5: size 1879610 MB<br>
Extending Allocation Map<br>
Checking Allocation Map for storage pool fc_ssd400G<br>
55 % complete on Thu Jan 5 14:43:31 2017<br>
Lost connection to file system daemon.<br>
mmadddisk: tsadddisk failed.<br>
Verifying file system configuration information ...<br>
mmadddisk: File system home has some disks that are in a non-ready state.<br>
mmadddisk: Propagating the cluster configuration data to all<br>
affected nodes. This is an asynchronous process.<br>
mmadddisk: Command failed. Examine previous error messages to determine cause.<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Had to use -v no (this failed once before). Anyhow I next see:<br>
[root@cl002 ~]# mmgetstate -aL<br>
<br>
Node number Node name Quorum Nodes up Total nodes GPFS state Remarks
<br>
------------------------------<wbr>------------------------------<wbr>------------------------<br>
1 cl001 0 0 8 down quorum node<br>
2 cl002 5 6 8 active quorum node<br>
3 cl003 5 0 8 arbitrating quorum node<br>
4 cl004 5 6 8 active quorum node<br>
5 cl005 5 6 8 active quorum node<br>
6 cl006 5 6 8 active quorum node<br>
7 cl007 5 6 8 active quorum node<br>
8 cl008 5 6 8 active quorum node<br>
[root@cl002 ~]# mmlsdisk home<br>
disk driver sector failure holds holds <wbr> storage<br>
name type size group metadata data status availability pool<br>
------------ -------- ------ ----------- -------- ----- ------------- ------------ ------------<br>
r10f1e5 nsd 512 1001 No Yes allocmap add up fc_ssd400G
<br>
r6d2e8 nsd 512 1001 No Yes ready up fc_8T
<br>
r6d3e8 nsd 512 1001 No Yes ready up fc_8T<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Do all quorum node have to be up and participating to do these admin type operations?
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
<br>
<hr>
<font size="1" color="Gray" face="Arial"><br>
Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this
email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness
or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial
product.<br>
</font>
</div>
<br>______________________________<wbr>_________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br>
<br></blockquote></div><br></div>