<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Helvetica Neue";
panose-1:2 0 5 3 0 0 0 2 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1239292512;
mso-list-template-ids:907821414;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">If you have two clusters that are hard to merge, but you are facing the need to provide capacity for more writes, another option to consider would be to set up a filesystem on GL2 with an AFM relationship to the filesystem on the netapp
gpfs cluster for accessing older data and point clients to the new GL2 filesystem.<o:p></o:p></p>
<p class="MsoNormal">Some downsides to that approach include introducing a dependency on afm (and potential performance reduction) to get to older data. There may also be complications depending on how your filesets are laid out.<o:p></o:p></p>
<p class="MsoNormal">At some point when you have more capacity in 5.x cluster and/or are ready to move off netapp, you could use afm to prefetch all data into new filesystem. In theory, you could then (re)build nsd servers connected to netapp on 5.x and add
them to new cluster and use them for a separate pool or keep them as a separate 5.x cluster.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best,<o:p></o:p></p>
<p class="MsoNormal">Chris<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black"><gpfsug-discuss-bounces@spectrumscale.org> on behalf of "Dorigo Alvise (PSI)" <alvise.dorigo@psi.ch><br>
<b>Reply-To: </b>gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Date: </b>Thursday, December 5, 2019 at 9:50 AM<br>
<b>To: </b>gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Subject: </b>Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div id="divtagdefaultwrapper">
<p><span style="font-size:12.0pt;color:black">This is a quite critical storage for data taking. It is not easy to update to GPFS5 because in that facility we have very short shutdown period. Thank you for pointing out that 4.2.3. But the entire storage will
be replaced in the future; at the moment we just need to expand it to survive for a while.<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
<p><span style="font-size:12.0pt;color:black">This merge seems quite tricky to implement and I haven't seen consistent opinions among the people that kindly answered. According to Jan Frode, Kaplan and T. Perry it should be possible, in principle, to do the
merge... Other people suggest a remote mount, which is not a solution for my use case. Other suggest not to do that...<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> A<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Daniel Kidger <daniel.kidger@uk.ibm.com><br>
<b>Sent:</b> Thursday, December 5, 2019 11:24:08 AM<br>
<b>To:</b> gpfsug main discussion list<br>
<b>Subject:</b> Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">One additional question to ask is : what are your long term plans for the 4.2.3 Spectrum Scake cluster? Do you expect to upgrade it to version 5.x (hopefully before 4.2.3 goes out of support)?
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Also I assume your Netapp hardware is the standard Netapp block storage, perhaps based on their standard 4U60 shelves daisy-chained together?<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Daniel<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">_________________________________________________________<br>
<b>Daniel Kidger</b><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">IBM Technical Sales Specialist<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Spectrum Scale, Spectrum Discover and IBM Cloud Object Store<br>
<br>
<a href="tel:+44-7818%20522%20266">+</a><a href="tel:+44-7818%20522%20266">44-(0)7818 522 266</a> <br>
<a href="mailto:daniel.kidger@uk.ibm.com">daniel.kidger@uk.ibm.com</a> <o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="264" style="width:2.75in;border-collapse:collapse;word-wrap: break-word">
<tbody>
<tr>
<td width="88" style="width:66.0pt;padding:.75pt .75pt .75pt .75pt;overflow:hidden">
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youracclaim.com_badges_687cf790-2Dfe65-2D4a92-2Db129-2Dd23ae41862ac_public-5Furl&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=GyO9ZkmILa6t9iQngf54AC7_OyD6tVoM1iVXGCL0Q6Q&e=" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:black"><img border="0" id="_x0000_i1034" src="https://images.youracclaim.com/images/c49300ae-d13e-4071-90f5-15f59d199c9e/IBM%2BVolunteers%2BGold%2Bv6.png" alt="https://images.youracclaim.com/images/c49300ae-d13e-4071-90f5-15f59d199c9e/IBM%2BVolunteers%2BGold%2Bv6.png"></span></span></a><o:p></o:p></p>
</td>
<td width="88" style="width:66.0pt;padding:.75pt .75pt .75pt .75pt;overflow:hidden">
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youracclaim.com_badges_8153c6a7-2D3e02-2D40be-2D87ee-2D24e27ae9459c_public-5Furl&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=9lvLFdrT5egMh5VvUB7zgC8TFl6o6vp8MXyJDKQJdr0&e=" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:black"><img border="0" id="_x0000_i1033" src="https://images.youracclaim.com/images/f2539224-f951-46b4-b376-b88f21c2be98/IBM-Selling-Certification---Level-1.png" alt="https://images.youracclaim.com/images/f2539224-f951-46b4-b376-b88f21c2be98/IBM-Selling-Certification---Level-1.png"></span></span></a><o:p></o:p></p>
</td>
<td width="88" style="width:66.0pt;padding:.75pt .75pt .75pt .75pt;overflow:hidden">
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youracclaim.com_badges_78197e2c-2D4277-2D4ec9-2D808b-2Dad6abe1e1b16_public-5Furl&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=2RPqu7daeHgCxrFYfDKnKWo1FfGC9Tx6UKqtxrFifzY&e=" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:black"><img border="0" id="_x0000_i1032" src="https://images.youracclaim.com/images/ea52b12f-97ac-4e72-8d24-b0ced8054e7d/Storage%2BTechnical%2BV1.png" alt="https://images.youracclaim.com/images/ea52b12f-97ac-4e72-8d24-b0ced8054e7d/Storage%2BTechnical%2BV1.png"></span></span></a><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt">On 5 Dec 2019, at 09:29, Dorigo Alvise (PSI) <alvise.dorigo@psi.ch> wrote:<o:p></o:p></p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div id="divtagdefaultwrapper">
<p><span style="font-size:12.0pt;color:black">Thank Anderson for the material. In principle our idea was to scratch the filesystem in the GL2, put its NSD on a dedicated pool and then merge it into the Filesystem which would remain on V4. I do not want to create
a FS in the GL2 but use its space to expand the space of the other cluster.<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> A<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Anderson Ferreira Nobre <anobre@br.ibm.com><br>
<b>Sent:</b> Wednesday, December 4, 2019 3:07:18 PM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hi Dorigo,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">From point of view of cluster administration I don't think it's a good idea to have hererogeneous cluster. There are too many diferences between V4 and V5. And much probably
many of enhancements of V5 you won't take advantage. One example is the new filesystem layout in V5. And at this moment the way to migrate the filesystem is create a new filesystem in V5 with the new layout and migrate the data. That is inevitable. I have
seen clients saying that they don't need all that enhancements, but the true is when you face performance issue that is only addressable with the new features someone will raise the question why we didn't consider that in the beginning.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Use this time to review if it would be better to change the block size of your fileystem. There's a script called filehist in /usr/lpp/mmfs/samples/debugtools to create a histogram
of files in your current filesystem. Here's the link with additional information:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ibm.com_developerworks_community_wikis_home-3Flang-3Den-23-21_wiki_General-2520Parallel-2520File-2520System-2520-28GPFS-29_page_Data-2520and-2520Metadata&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=Nxokewkn7kpufxAbH2mh5o9yKPNVufApADUYH37zQMg&e=">https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20(GPFS)/page/Data%20and%20Metadata</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Different RAID configurations also brings unexpected performance behaviors. Unless you are planning create different pools and use ILM to manage the files in different pools.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">One last thing, it's a good idea to follow the recommended levels for Spectrum Scale:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ibm.com_support_pages_ibm-2Dspectrum-2Dscale-2Dsoftware-2Dversion-2Drecommendation-2Dpreventive-2Dservice-2Dplanning&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=gezVwyLvQg92OQjB0iZlfQrcgIw2QtueG-LmJD_d3SM&e=">https://www.ibm.com/support/pages/ibm-spectrum-scale-software-version-recommendation-preventive-service-planning</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Anyway, you are the system administrator, you know better than anyone how complex is to manage this cluster.<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="650" style="width:487.5pt">
<tbody>
<tr>
<td valign="bottom" style="padding:0in 0in 0in 0in">
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#5F5F5F">Abraços / Regards / Saludos,<o:p></o:p></span></p>
</div>
<p><span style="font-size:7.0pt;color:#5F5F5F"> <o:p></o:p></span></p>
<div style="margin-bottom:6.0pt">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:#888888">AndersonNobre</span></b><span style="font-size:7.0pt;color:#5F5F5F"><br>
Power and Storage Consultant<br>
IBM Systems Hardware Client Technical Team – IBM Systems Lab Services<br>
<br>
<img border="0" width="333" height="120" style="width:3.4687in;height:1.25in" id="_x0000_i1030" src="http://esign.webmaster.ibm.com/img/signature_lab_services.jpg" alt="community_general_lab_services"><o:p></o:p></span></p>
</div>
</td>
<td style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-size:7.0pt;color:#5F5F5F"> <o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;display:none"><o:p> </o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="650" style="width:487.5pt">
<tbody>
<tr>
<td colspan="3" style="padding:0in 0in 0in 0in">
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:7.0pt;color:#5F5F5F">
<hr size="0" width="100%" noshade="" style="color:dimgray" align="center">
</span></div>
</td>
</tr>
<tr>
<td valign="top" style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:7.0pt;font-family:"Arial",sans-serif;color:#466BB0">Phone:</span></b><span style="font-size:7.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">55-19-2132-4317<br>
</span><b><span style="font-size:7.0pt;font-family:"Arial",sans-serif;color:#466BB0">E-mail:</span></b><span style="font-size:7.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">
<a href="mailto:anobre@br.ibm.com" target="_blank"><span style="color:#5F5F5F">anobre@br.ibm.com</span></a><o:p></o:p></span></p>
</td>
<td valign="top" style="padding:0in 0in 0in 0in">
<p class="MsoNormal" align="right" style="text-align:right"><span style="font-size:7.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"><img border="0" id="_x0000_i1028" src="http://esign.webmaster.ibm.com/img/ibm3.gif" alt="IBM"><o:p></o:p></span></p>
</td>
<td style="padding:0in 0in 0in 0in"></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #AAAAAA 1.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">----- Original message -----<br>
From: "Dorigo Alvise (PSI)" <alvise.dorigo@psi.ch><br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
To: "gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
Cc:<br>
Subject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster<br>
Date: Wed, Dec 4, 2019 06:44<br>
<br>
<br>
<o:p></o:p></span></p>
<div id="divtagdefaultwrapper">
<p><span style="font-size:12.0pt;color:black">Thank you all for the answer. I try to recap my answers to your questions:<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<ol start="1" type="1">
<li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:12.0pt">the purpose is not to merge clusters "per se"; it is adding the GL2's 700TB raw space to the current filesystem provided by the GPFS/NetApp (which is running out of free space); of course I know well the heterogeneity of this
hypothetical system, so the GL2's NSD would go to a special pool; but in the end I need a unique namespace for files.
<o:p></o:p></span></li><li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:12.0pt">I do not want to do the opposite (mergin GPFS/NetApp into the GL2 cluster) because the former is in production and I cannot schedule long downtimes
<o:p></o:p></span></li><li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:12.0pt">All system have proper licensing of course; what does it means that I could loose IBM support ? if the support is for a failing disk drive I do not think so; if the support is for a "strange" behaviour of GPFS I can probably understand
<o:p></o:p></span></li><li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:12.0pt">NSD (in the NetApp system) are in their roles: what do you mean exactly ? there's RAIDset attached to servers that are actually NSD together with their attached LUN<o:p></o:p></span></li></ol>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"> Alvise<o:p></o:p></span></p>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<hr size="0" width="100%" align="center">
</span></div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Lyle Gayne <lgayne@us.ibm.com><br>
<b>Sent:</b> Tuesday, December 3, 2019 8:30:31 PM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Cc:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">For:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">- A NetApp system with hardware RAID<br>
- SpectrumScale 4.2.3-13 running on top of the NetApp <strong><span style="font-family:"Calibri",sans-serif">< --- Are these NSD servers in their GPFS roles (where Scale "runs on top"</span></strong>?<br>
- A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1)</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt">What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc.</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt">I'd like to ask the experts</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
1. </span><span style="font-size:12.0pt">whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
2. </span><span style="font-size:12.0pt">if yes, whether anyone already did something like this and what is the best strategy suggested</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
3. </span><span style="font-size:12.0pt">finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">......<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Some observations:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">1) Why do you want to MERGE the GL2 into a single cluster with the rest cluster, rather than simply allowing remote mount of the ESS servers by the other GPFS (NSD client) nodes?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">2) Interop of the mix of 4.2 and 5.0 levels should be allowed by our coexistence rules.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">3) Mixing x86 and Power, especially as NSD servers, should pose no issues. Having them as separate file systems (NetApp vs. ESS) means no concerns regarding varying architectures
within the same fs serving or failover scheme. Mixing such as compute nodes would mean some performance differences across the different clients, but you haven't described your compute (NSD client) details.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Lyle<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #AAAAAA 1.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">----- Original message -----<br>
From: "Tomer Perry" <TOMP@il.ibm.com><br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Cc:<br>
Subject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster<br>
Date: Tue, Dec 3, 2019 10:03 AM<br>
<br>
Hi,<br>
<br>
Actually, I believe that GNR is not a limiting factor here. <br>
mmexportfs and mmimportfs ( man mm??portfs) will export/import GNR configuration as well:<br>
"</span><span style="font-size:12.0pt;font-family:"Helvetica Neue";color:#2F2F2F">If the specified file system device is a IBM Spectrum Scale RAID-based file system, then all affected IBM Spectrum Scale RAID objects will be exported as well. This includes recovery
groups, declustered arrays, vdisks, and any other file systems that are based on these objects. For more information about IBM Spectrum Scale RAID, see <i>IBM Spectrum Scale RAID: Administration</i>.</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif">"<br>
<br>
OTOH, I suspect that due to the version mismatch, it wouldn't work - since I would assume that the cluster config version is to high for the NetApp based cluster.<br>
I would also suspect that the filesystem version on the ESS will be different.<br>
<br>
<br>
Regards,<br>
<br>
Tomer Perry<br>
Scalable I/O Development (Spectrum Scale)<br>
email: tomp@il.ibm.com<br>
1 Azrieli Center, Tel Aviv 67021, Israel<br>
Global Tel: +1 720 3422758<br>
Israel Tel: +972 3 9188625<br>
Mobile: +972 52 2554625<br>
<br>
<br>
<br>
<br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">From: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Olaf Weiser" <olaf.weiser@de.ibm.com></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">To: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">03/12/2019 16:54</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Subject: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">[EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Sent by: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="center">
</span></div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
<br>
Hallo<br>
"merging" 2 different GPFS cluster into one .. is not possible ..<br>
for sure you can do "nested" mounts .. .but that's most likely not, what you want to do ..
<br>
<br>
if you want to add a GL2 (or any other ESS) ..to an existing (other) cluster... - you can't preserve ESS's RG definitions...<br>
you need to create the RGs after adding the IO-nodes to the existing cluster... <br>
<br>
so if you got a new ESS.. (no data on it) .. simply unconfigure cluster .. .. add the nodes to your existing cluster.. and then start configuring the RGs<br>
<br>
<br>
<br>
<br>
<br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">From: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Dorigo Alvise (PSI)" <alvise.dorigo@psi.ch></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">To: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">12/03/2019 09:35 AM</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Subject: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">[EXTERNAL] [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Sent by: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="center">
</span></div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
<br>
</span><span style="font-size:12.0pt">Hello everyone,<br>
I have:<br>
- A NetApp system with hardware RAID<br>
- SpectrumScale 4.2.3-13 running on top of the NetApp<br>
- A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1)</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt">What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc.</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt">I'd like to ask the experts</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
1. </span><span style="font-size:12.0pt">whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
2. </span><span style="font-size:12.0pt">if yes, whether anyone already did something like this and what is the best strategy suggested</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
3. </span><span style="font-size:12.0pt">finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt">Thank you very much,</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
</span><span style="font-size:12.0pt"> Alvise Dorigo</span><tt><span style="font-size:12.0pt">_______________________________________________</span></tt><span style="font-size:12.0pt;font-family:"Courier New""><br>
<tt>gpfsug-discuss mailing list</tt><br>
<tt>gpfsug-discuss at spectrumscale.org</tt></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><tt><u><span style="font-size:12.0pt;color:blue"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=0HtvaNxKjx89vOc40bGiS9T0qIzNKCK7AamkyFGM91I&e=" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span></u></tt><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
<br>
</span><tt><span style="font-size:12.0pt">_______________________________________________</span></tt><span style="font-size:12.0pt;font-family:"Courier New""><br>
<tt>gpfsug-discuss mailing list</tt><br>
<tt>gpfsug-discuss at spectrumscale.org</tt></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><tt><span style="font-size:12.0pt"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=0HtvaNxKjx89vOc40bGiS9T0qIzNKCK7AamkyFGM91I&e=" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span></tt><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
<br>
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=0HtvaNxKjx89vOc40bGiS9T0qIzNKCK7AamkyFGM91I&e=" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMGaQ&c=C9X8xNkG_lwP_-eFHTGejw&r=DopWM-bvfskhBn2zeglfyyw5U2pumni6m_QzQFYFepU&m=Qny5Z-9GdZHIJItr9sIUwnUtBPfbxuyiVDLWAahyMo4&s=0HtvaNxKjx89vOc40bGiS9T0qIzNKCK7AamkyFGM91I&e=" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number 741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU<o:p></o:p></p>
</div>
</div>
<hr>
<div style="font-size:7.5pt; font-family: arial; font-style:normal; font-weight:normal; ">
This message is for the recipient’s use only, and may contain confidential, privileged or protected information. Any unauthorized use or dissemination of this communication is prohibited. If you received this message in error, please immediately notify the
sender and destroy all copies of this message. The recipient should check this email and any attachments for the presence of viruses, as we accept no liability for any damage caused by any virus transmitted by this email.</div>
</body>
</html>