<div dir="ltr">It does sound like "mmvdisk rg change --restart" is the "varyon" command you're looking for.. but it's not clear why it's failing. I would start by looking at if there are any lower level issues with your cluster. Are your nodes healthy on a GPFS-level? "mmnetverify -N all" says network is OK ? "mmhealth node show -N all" not indicating any issues ? Check mmfs.log.latest ?</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 24, 2023 at 1:41 PM Walter Sklenka <<a href="mailto:Walter.Sklenka@edv-design.at">Walter.Sklenka@edv-design.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="msg1953039553672333482">
<div lang="DE-AT" style="overflow-wrap: break-word;">
<div class="m_4673065097918922215WordSection1">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:Arial,sans-serif;color:black">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b><br>
</span><b><i><span style="font-size:8pt;font-family:Arial,sans-serif;color:gray">Technical Consultant</span></i></b><span style="color:black">
</span><span style="color:rgb(33,33,33)"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif;color:black"> </span><span style="color:rgb(33,33,33)"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:12pt"><span style="font-size:8pt;font-family:Arial,sans-serif;color:black">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: <a href="mailto:sklenka@edv-design.at" target="_blank">sklenka@edv-design.at</a></span><span style="font-size:8pt;color:black"><br>
</span><span style="font-size:8pt;font-family:Arial,sans-serif;color:black">Internet:
<a href="http://www.edv-design.at/" target="_blank">www.edv-design.at</a><u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div style="border-width:1pt medium medium;border-style:solid none none;border-color:rgb(225,225,225) currentcolor currentcolor;padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Walter Sklenka
<br>
<b>Sent:</b> Donnerstag, 24. August 2023 12:02<br>
<b>To:</b> '<a href="mailto:gpfsug-discuss-request@gpfsug.org" target="_blank">gpfsug-discuss-request@gpfsug.org</a>' <<a href="mailto:gpfsug-discuss-request@gpfsug.org" target="_blank">gpfsug-discuss-request@gpfsug.org</a>><br>
<b>Subject:</b> FW: ESS 3500-C5 : rg has resigned permanently<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span lang="EN-US">Hi !<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Does someone eventually have experience with ESS 3500 ( no hybrid config, only NLSAS with 5 enclosures )
<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">We have issues with a shared recoverygroup. After creating it we made a test of setting only one node active (mybe not an optimal idea)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">But since then the recoverygroup is down<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">We have created a PMR but do not get any response until now.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">The rg has no vdisks of any filesystem <u></u>
<u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9pt;font-family:"Courier New";color:rgb(22,22,22);letter-spacing:0.1pt;background:repeat rgb(243,242,242)">[gpfsadmin@hgess02-m ~]$ ^C</span><span lang="EN-US" style="font-size:9pt;font-family:"Courier New";color:rgb(22,22,22);letter-spacing:0.1pt"><br>
<span style="background:repeat rgb(243,242,242)">[gpfsadmin@hgess02-m ~]$ sudo mmvdisk rg change --rg ess3500_hgess02_n1_hs_hgess02_n2_hs --restart</span><br>
<span style="background:repeat rgb(243,242,242)">mmvdisk: </span><br>
<span style="background:repeat rgb(243,242,242)">mmvdisk: </span><br>
<span style="background:repeat rgb(243,242,242)">mmvdisk: Unable to reset server list for recovery group 'ess3500_hgess02_n1_hs_hgess02_n2_hs'.</span><br>
<span style="background:repeat rgb(243,242,242)">mmvdisk: Command failed. </span></span><span style="font-size:9pt;font-family:"Courier New";color:rgb(22,22,22);letter-spacing:0.1pt;background:repeat rgb(243,242,242)">Examine previous error messages to determine cause.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">We also tried <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9pt;font-family:"Courier New";color:rgb(22,22,22);letter-spacing:0.1pt;background:repeat rgb(243,242,242)">2023-08-21_16:57:26.174+0200: [I] Command: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l root hgess02-n2-hs.invalid</span><span lang="EN-US" style="font-size:9pt;font-family:"Courier New";color:rgb(22,22,22);letter-spacing:0.1pt"><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.201+0200: [I] Recovery group ess3500_hgess02_n1_hs_hgess02_n2_hs has resigned permanently</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.201+0200: [E] Command: err 2: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l root hgess02-n2-hs.invalid</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.201+0200: Specified entity, such as a disk or file system, does not exist.</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.207+0200: [I] Command: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l LG001 hgess02-n2-hs.invalid.</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.207+0200: [E] Command: err 212: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l LG001 hgess02-n2-hs.invalid</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.207+0200: The current file system manager failed and no new manager will be appointed. This may cause nodes mounting the file system to experience mount failures.</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.213+0200: [I] Command: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l LG002 hgess02-n2-hs.invalid</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.213+0200: [E] Command: err 212: tsrecgroupserver ess3500_hgess02_n1_hs_hgess02_n2_hs -f -l LG002 hgess02-n2-hs.invalid</span><br>
<span style="background:repeat rgb(243,242,242)">2023-08-21_16:57:26.213+0200: The current file system manager failed and no new manager will be appointed. This may cause nodes mounting the file system to experience mount failures.</span></span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">For us it is crucial to know what we can do if theis happens again ( it has no vdisks yet so it is not critical ).</span><span lang="EN-US" style="font-size:12pt;font-family:"IBM Plex Sans",sans-serif;color:rgb(22,22,22);border:1pt windowtext;padding:0cm;background:repeat white">
</span><span lang="EN-US" style="font-size:12pt;font-family:"IBM Plex Sans",sans-serif;color:black;border:1pt windowtext;padding:0cm;background:repeat white"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Do you know: is there a non documented way to “vary on”, or activate a recoverygroup again?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">The doc :<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://www.ibm.com/docs/en/ess/6.1.6_lts?topic=rgi-recovery-group-issues-shared-recovery-groups-in-ess" target="_blank">https://www.ibm.com/docs/en/ess/6.1.6_lts?topic=rgi-recovery-group-issues-shared-recovery-groups-in-ess</a><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">tells to mmshutdown and mmstartup, but the RGCM does say nothing<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">When trying to execute any vdisk command it only says “rg down”, no idea how we could recover from that without deleting the rg ( I hope it will never happen, when we have vdisks on it<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Have a nice day<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Walter <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:Arial,sans-serif;color:black">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b><br>
</span><b><i><span style="font-size:8pt;font-family:Arial,sans-serif;color:gray">Technical Consultant</span></i></b><span style="color:black">
</span><span style="color:rgb(33,33,33)"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif;color:black"> </span><span style="color:rgb(33,33,33)"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:12pt"><span style="font-size:8pt;font-family:Arial,sans-serif;color:black">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: </span><a href="mailto:sklenka@edv-design.at" target="_blank"><span style="font-size:8pt;font-family:Arial,sans-serif">sklenka@edv-design.at</span></a><span style="font-size:8pt;color:black"><br>
</span><span style="font-size:8pt;font-family:Arial,sans-serif;color:black">Internet:
</span><a href="http://www.edv-design.at/" target="_blank"><span style="font-size:8pt;font-family:Arial,sans-serif">www.edv-design.at</span></a><span style="font-size:8pt;font-family:Arial,sans-serif;color:black"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://gpfsug.org" rel="noreferrer" target="_blank">gpfsug.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a><br>
</div></blockquote></div>