<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">>Starting GPFS 5.1.4, you can use the CCR archive to restore the local node (the node that is issuing the mmsdrrestore command) beside restoring the entire cluster.<br>
<br>
This is great addition, but how does the security model work?<br>
ie. How do the other cluster nodes know that this is a newly re-installed node can be trusted and is not a rogue node trying to gain cluster membership by a backdoor?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Daniel<br>
<br>
</span><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> on behalf of Truong Vu <truongv@us.ibm.com><br>
<b>Date: </b>Saturday, 30 July 2022 at 01:35<br>
<b>To: </b>gpfsug-discuss@gpfsug.org <gpfsug-discuss@gpfsug.org><br>
<b>Subject: </b>Re: [gpfsug-discuss] add local nsd back to cluster?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Starting GPFS 5.1.4, you can use the CCR archive to restore the local node (the node that is issuing the mmsdrrestore command) beside restoring the entire cluster.<br>
<br>
Prior to GPFS5.1.4, as the error message reviewed, you can only use the CCR archive to restore the entire cluster. GPFS must be down any node that is being restored.
<br>
If is a good node in the cluster, use the -p option<br>
<br>
-p NodeName<br>
Specifies the node from which to obtain a valid GPFS<br>
configuration file. The node must be either the primary<br>
configuration server or a node that has a valid backup<br>
copy of the mmsdrfs file. If this parameter is not<br>
specified, the command uses the configuration file on<br>
the node from which the command is issued.<br>
<br>
Thanks,<br>
Tru.<br>
<br>
On 7/29/22, 12:51 PM, "gpfsug-discuss on behalf of gpfsug-discuss-request@gpfsug.org" <gpfsug-discuss-bounces@gpfsug.org on behalf of gpfsug-discuss-request@gpfsug.org> wrote:<br>
<br>
Send gpfsug-discuss mailing list submissions to<br>
gpfsug-discuss@gpfsug.org<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<br>
or, via email, send a message with subject or body 'help' to<br>
gpfsug-discuss-request@gpfsug.org<br>
<br>
You can reach the person managing the list at<br>
gpfsug-discuss-owner@gpfsug.org<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of gpfsug-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: add local nsd back to cluster? (shao feng)<br>
2. Re: add local nsd back to cluster? (Stephen Ulmer)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 29 Jul 2022 23:54:24 +0800<br>
From: shao feng <shaof777@gmail.com><br>
To: gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
Subject: Re: [gpfsug-discuss] add local nsd back to cluster?<br>
Message-ID:<br>
<CANiV0ORjKzbyKqLvHgQEkPKo9Y--ptPRxfPjXpJBvkQmukqCgA@mail.gmail.com><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Thanks Olaf<br>
<br>
I've setup the mmsdr backup as<br>
<a href="https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=exits-mmsdrbackup-user-exit">https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=exits-mmsdrbackup-user-exit</a> ,<br>
since my cluster is CCR enabled, it generate a CCR backup file,<br>
but when trying to restore from this file, it require quorum nodes to<br>
shutdown? Is it possible to restore without touching quorum nodes?<br>
<br>
[root@tofail ~]# mmsdrrestore -F<br>
CCRBackup.986.2022.07.29.23.06.19.myquorum.tar.gz<br>
Restoring a CCR backup archive is a cluster-wide operation.<br>
The -a flag is required.<br>
mmsdrrestore: Command failed. Examine previous error messages to determine<br>
cause.<br>
<br>
[root@tofail ~]# mmsdrrestore -F<br>
CCRBackup.986.2022.07.29.23.06.19.myquorum.tar.gz -a<br>
Restoring CCR backup<br>
Verifying that GPFS is inactive on quorum nodes<br>
mmsdrrestore: GPFS is still active on myquorum<br>
mmsdrrestore: Unexpected error from mmsdrrestore: CCR restore failed.<br>
Return code: 192<br>
mmsdrrestore: Command failed. Examine previous error messages to determine<br>
cause.<br>
<br>
<br>
On Thu, Jul 28, 2022 at 3:14 PM Olaf Weiser <olaf.weiser@de.ibm.com> wrote:<br>
<br>
><br>
><br>
> Hi -<br>
> assuming, you'll run it withou ECE ?!? ... just with replication on the<br>
> file system level<br>
> ba aware, every time a node goes offline, you 'll have to restart the<br>
> disks in your filesystem .. This causes a complete scan of the meta data to<br>
> detect files with missing updates / replication<br>
><br>
><br>
> apart from that to your Q :<br>
> you may consider to backup mmsdr<br>
> additionally, take a look to mmsdrrestore, in case you want to restore a<br>
> nodes's SDR configuration<br>
><br>
> quick and dirty.. save the content of /var/mmfs may also help you<br>
><br>
> during the node is "gone".. of course.. the disk is down , after restore<br>
> of SDR / node's config .. it should be able to start ..<br>
> the rest runs as usual<br>
><br>
><br>
><br>
> ------------------------------<br>
> *Von:* gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> im Auftrag von<br>
> shao feng <shaof777@gmail.com><br>
> *Gesendet:* Donnerstag, 28. Juli 2022 09:02<br>
> *An:* gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
> *Betreff:* [EXTERNAL] [gpfsug-discuss] add local nsd back to cluster?<br>
><br>
> Hi all, I am planning to implement a cluster with a bunch of old x86<br>
> machines, the disks are not connected to nodes via the SAN network, instead<br>
> each x86 machine has some local attached disks. The question is regarding<br>
> node failure, for example<br>
> <br>
> Hi all,<br>
><br>
> I am planning to implement a cluster with a bunch of old x86 machines,<br>
> the disks are not connected to nodes via the SAN network, instead each x86<br>
> machine has some local attached disks.<br>
> The question is regarding node failure, for example only the operating<br>
> system disk fails and the nsd disks are good. In that case I plan to<br>
> replace the failing OS disk with a new one and install the OS on it and<br>
> re-attach these nsd disks to that node, my question is: will this work? how<br>
> can I add a nsd back to the cluster without restoring data from other<br>
> replicas since the data/metadata is actually not corrupted on nsd.<br>
><br>
> Best regards,<br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at gpfsug.org<br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20220729/1fc0e167/attachment-0001.htm ><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Fri, 29 Jul 2022 12:48:44 -0400<br>
From: Stephen Ulmer <ulmer@ulmer.org><br>
To: gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
Subject: Re: [gpfsug-discuss] add local nsd back to cluster?<br>
Message-ID: <1DEB036E-AA3A-4498-A5B9-B66078EC87A9@ulmer.org><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
If there are cluster nodes up, restore from the running nodes instead of the file. I think it?s -p, but look at the manual page.<br>
<br>
-- <br>
Stephen Ulmer<br>
<br>
Sent from a mobile device; please excuse auto-correct silliness.<br>
<br>
> On Jul 29, 2022, at 11:20 AM, shao feng <shaof777@gmail.com> wrote:<br>
> <br>
> ?<br>
> Thanks Olaf<br>
> <br>
> I've setup the mmsdr backup as <a href="https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=exits-mmsdrbackup-user-exit">https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=exits-mmsdrbackup-user-exit</a> , since my cluster is CCR enabled, it generate a CCR
backup file,<br>
> but when trying to restore from this file, it require quorum nodes to shutdown? Is it possible to restore without touching quorum nodes?<br>
> <br>
> [root@tofail ~]# mmsdrrestore -F CCRBackup.986.2022.07.29.23.06.19.myquorum.tar.gz<br>
> Restoring a CCR backup archive is a cluster-wide operation.<br>
> The -a flag is required.<br>
> mmsdrrestore: Command failed. Examine previous error messages to determine cause.<br>
> <br>
> [root@tofail ~]# mmsdrrestore -F CCRBackup.986.2022.07.29.23.06.19.myquorum.tar.gz -a<br>
> Restoring CCR backup<br>
> Verifying that GPFS is inactive on quorum nodes<br>
> mmsdrrestore: GPFS is still active on myquorum<br>
> mmsdrrestore: Unexpected error from mmsdrrestore: CCR restore failed. Return code: 192<br>
> mmsdrrestore: Command failed. Examine previous error messages to determine cause.<br>
> <br>
> <br>
>> On Thu, Jul 28, 2022 at 3:14 PM Olaf Weiser <olaf.weiser@de.ibm.com> wrote:<br>
>> <br>
>> <br>
>> Hi - <br>
>> assuming, you'll run it withou ECE ?!? ... just with replication on the file system level
<br>
>> ba aware, every time a node goes offline, you 'll have to restart the disks in your filesystem .. This causes a complete scan of the meta data to detect files with missing updates / replication<br>
>> <br>
>> <br>
>> apart from that to your Q :<br>
>> you may consider to backup mmsdr <br>
>> additionally, take a look to mmsdrrestore, in case you want to restore a nodes's SDR configuration
<br>
>> <br>
>> quick and dirty.. save the content of /var/mmfs may also help you <br>
>> <br>
>> during the node is "gone".. of course.. the disk is down , after restore of SDR / node's config .. it should be able to start ..
<br>
>> the rest runs as usual <br>
>> <br>
>> <br>
>> <br>
>> Von: gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> im Auftrag von shao feng <shaof777@gmail.com><br>
>> Gesendet: Donnerstag, 28. Juli 2022 09:02<br>
>> An: gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
>> Betreff: [EXTERNAL] [gpfsug-discuss] add local nsd back to cluster?<br>
>> <br>
>> This Message Is From an External Sender<br>
>> This message came from outside your organization.<br>
>> <br>
>> Hi all,<br>
>> <br>
>> I am planning to implement a cluster with a bunch of old x86 machines, the disks are not connected to nodes via the SAN network, instead each x86 machine has some local attached disks.<br>
>> The question is regarding node failure, for example only the operating system disk fails and the nsd disks are good. In that case I plan to replace the failing OS disk with a new one and install the OS on it and re-attach these nsd disks to that node,
my question is: will this work? how can I add a nsd back to the cluster without restoring data from other replicas since the data/metadata is actually not corrupted on nsd.<br>
>> <br>
>> Best regards,<br>
>> _______________________________________________<br>
>> gpfsug-discuss mailing list<br>
>> gpfsug-discuss at gpfsug.org<br>
>> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at gpfsug.org<br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20220729/1c773ee2/attachment.htm ><br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<br>
<br>
<br>
------------------------------<br>
<br>
End of gpfsug-discuss Digest, Vol 126, Issue 21<br>
***********************************************<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
<o:p></o:p></span></p>
</div>
</div>
</body>
</html>