<div><div dir="auto">It doesn’t seem to be documented anywhere, but we can add ESS to nok-ESS clusters. It’s mostly just following the QDG, skipping the gssgencluster step.</div></div><div dir="auto"><br></div><div dir="auto">Just beware that it will take down your current cluster when doing the first «gssgenclusterrgs». This is to change quite a few config settings — it recently caugth me by surprise :-/</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto">Involve IBM lab services, and we should be able to help :-)</div><div dir="auto"> </div><div dir="auto"><br></div><div dir="auto"> -jf</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">ons. 3. apr. 2019 kl. 21:35 skrev Prasad Surampudi <<a href="mailto:prasad.surampudi@theatsgroup.com">prasad.surampudi@theatsgroup.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Actually, we have a SAS Grid - Scale cluster with V7000 and Flash storage. We also have protocol nodes for SMB access to SAS applications/users. Now, we are planning to gradually move our cluster from V7000/Flash to ESS and retire V7Ks. So, when we grow our
filesystem, we are thinking of adding an ESS as an additional block of storage instead of adding another V7000. Definitely we'll keep the ESS Disk Enclosures in a separate GPFS pool in the same filesystem, but can't create a new filesystem as we want to have
single name space for our SMB Shares. Also, we'd like keep all our existing compute, protocol, and NSD servers all in the same scale cluster along with ESS IO nodes and EMS. When I looked at ESS commands, I dont see an option of adding ESS nodes to existing
cluster like mmaddnode or similar commands. So, just wondering how we could add ESS IO nodes to existing cluster like any other node..is running mmaddnode command on ESS possible? Also, looks like it's against the IBMs recommendation of separating the Storage,
Compute and Protocol nodes into their own scale clusters and use cross-cluster filesystem mounts..any comments/suggestions?</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_3579393006593949541signature">
<div>
<div style="text-align:start;background-color:rgb(255,255,255)">
<div>
<div>
<p><font color="#000000" face="Tahoma, sans-serif"><span style="font-size:14px">Prasad Surampudi</span></font></p>
<p><font color="#000000" face="Tahoma, sans-serif"><span style="font-size:14px">The ATS Group</span></font></p>
</div>
</div>
<div style="font-family:Arial,Helvetica,sans-serif;color:rgb(34,34,34);font-size:small">
<div></div>
</div>
<div style="font-family:Arial,Helvetica,sans-serif;color:rgb(34,34,34);font-size:small">
<p><u></u></p>
</div>
</div>
<div style="font-family:Arial,Helvetica,sans-serif;color:rgb(34,34,34);font-size:small;text-align:start;background-color:rgb(255,255,255)">
<div><br>
</div>
</div>
<br>
</div>
</div>
<div id="m_3579393006593949541appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="m_3579393006593949541divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> <<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of <a href="mailto:gpfsug-discuss-request@spectrumscale.org" target="_blank">gpfsug-discuss-request@spectrumscale.org</a>
<<a href="mailto:gpfsug-discuss-request@spectrumscale.org" target="_blank">gpfsug-discuss-request@spectrumscale.org</a>><br>
<b>Sent:</b> Wednesday, April 3, 2019 2:54 PM<br>
<b>To:</b> <a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>
<b>Subject:</b> gpfsug-discuss Digest, Vol 87, Issue 4</font>
<div> </div>
</div>
<div class="m_3579393006593949541BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="m_3579393006593949541PlainText">Send gpfsug-discuss mailing list submissions to<br>
<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:gpfsug-discuss-request@spectrumscale.org" target="_blank">gpfsug-discuss-request@spectrumscale.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:gpfsug-discuss-owner@spectrumscale.org" target="_blank">gpfsug-discuss-owner@spectrumscale.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of gpfsug-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Adding ESS to existing Scale Cluster (Sanchez, Paul)<br>
2. New ESS install - Network adapter down level (Oesterlin, Robert)<br>
3. Re: New ESS install - Network adapter down level<br>
(Jan-Frode Myklebust)<br>
4. Re: New ESS install - Network adapter down level<br>
(Stephen R Buchanan)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Wed, 3 Apr 2019 16:41:32 +0000<br>
From: "Sanchez, Paul" <<a href="mailto:Paul.Sanchez@deshaw.com" target="_blank">Paul.Sanchez@deshaw.com</a>><br>
To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br>
Subject: Re: [gpfsug-discuss] Adding ESS to existing Scale Cluster<br>
Message-ID: <<a href="mailto:fbde9b5a4d364a41bc5c2c1eeb2c0438@deshaw.com" target="_blank">fbde9b5a4d364a41bc5c2c1eeb2c0438@deshaw.com</a>><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
> note though you can't have GNR based vdisks (ESS/DSS-G) in the same storage pool.<br>
<br>
At one time there was definitely a warning from IBM in the docs about not mixing big-endian and little-endian GNR in the same cluster/filesystem. But at least since Nov 2017, IBM has published videos showing clusters containing both. (In my opinion, they
had to support this because they changed the endian-ness of the ESS from BE to LE.)
<br>
<br>
I don't know about all ancillary components (e.g. GUI) but as for Scale itself, I can confirm that filesystems can contain NSDs which are provided by ESS(BE), ESS(LE), GSS, and DSS in all combinations, along with SAN storage based NSD servers. We typically
do rolling upgrades of GNR building blocks by adding blocks to an existing cluster, emptying and removing the existing blocks, upgrading those in isolation, then repeating with the next cluster. As a result, we have had every combination in play at some point
in time. Care just needs to be taken with nodeclass naming and mmchconfig parameters. (We derive the correct params for each new building block from its final config after upgrading/testing it in isolation.)<br>
<br>
-Paul<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> <<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>> On Behalf Of Simon Thompson<br>
Sent: Wednesday, April 3, 2019 12:18 PM<br>
To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br>
Subject: Re: [gpfsug-discuss] Adding ESS to existing Scale Cluster<br>
<br>
We have DSS-G (Lenovo equivalent) in the same cluster as other SAN/IB storage (IBM, DDN). But we don't have them in the same file-system.<br>
<br>
In theory as a different pool it should work, note though you can't have GNR based vdisks (ESS/DSS-G) in the same storage pool.<br>
<br>
And if you want to move to new block size or v5 variable sunblocks then you are going to have to have a new filesystem and copy data. So it depends what your endgame is really. We just did such a process and one of my colleagues is going to talk about it at
the London user group in May.<br>
<br>
Simon<br>
________________________________________<br>
From: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> [<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>] on behalf of <a href="mailto:prasad.surampudi@theatsgroup.com" target="_blank">prasad.surampudi@theatsgroup.com</a> [<a href="mailto:prasad.surampudi@theatsgroup.com" target="_blank">prasad.surampudi@theatsgroup.com</a>]<br>
Sent: 03 April 2019 17:12<br>
To: <a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>
Subject: [gpfsug-discuss] Adding ESS to existing Scale Cluster<br>
<br>
We are planning to add an ESS GL6 system to our existing Spectrum Scale cluster. Can the ESS nodes be added to existing scale cluster without changing existing cluster name? Or do we need to create a new scale cluster with ESS and import existing filesystems
into the new ESS cluster?<br>
<br>
Prasad Surampudi<br>
Sr. Systems Engineer<br>
The ATS Group<br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Wed, 3 Apr 2019 17:25:54 +0000<br>
From: "Oesterlin, Robert" <<a href="mailto:Robert.Oesterlin@nuance.com" target="_blank">Robert.Oesterlin@nuance.com</a>><br>
To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br>
Subject: [gpfsug-discuss] New ESS install - Network adapter down level<br>
Message-ID: <<a href="mailto:66070BCC-1D30-48E5-B0E7-0680865F0E4D@nuance.com" target="_blank">66070BCC-1D30-48E5-B0E7-0680865F0E4D@nuance.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Any insight on what command I need to fix this? It?s the only error I have when running gssinstallcheck.<br>
<br>
[ERROR] Network adapter MT4115 firmware: found 12.23.1020 expected 12.23.8010, net adapter count: 4<br>
<br>
<br>
Bob Oesterlin<br>
Sr Principal Storage Engineer, Nuance<br>
507-269-0413<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/42850e8f/attachment-0001.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/42850e8f/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Wed, 3 Apr 2019 20:11:45 +0200<br>
From: Jan-Frode Myklebust <<a href="mailto:janfrode@tanso.net" target="_blank">janfrode@tanso.net</a>><br>
To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br>
Subject: Re: [gpfsug-discuss] New ESS install - Network adapter down<br>
level<br>
Message-ID:<br>
<CAHwPatjx-r7EHoV_PuB1XMGyGgQomz6KtLS=<a href="mailto:FrPJyB-36ZJX7w@mail.gmail.com" target="_blank">FrPJyB-36ZJX7w@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Have you tried:<br>
<br>
updatenode nodename -P gss_ofed<br>
<br>
But, is this the known issue listed in the qdg?<br>
<br>
<br>
<a href="https://www.ibm.com/support/knowledgecenter/SSYSP8_5.3.2/ess_qdg.pdf" target="_blank">https://www.ibm.com/support/knowledgecenter/SSYSP8_5.3.2/ess_qdg.pdf</a><br>
<br>
<br>
-jf<br>
<br>
ons. 3. apr. 2019 kl. 19:26 skrev Oesterlin, Robert <<br>
<a href="mailto:Robert.Oesterlin@nuance.com" target="_blank">Robert.Oesterlin@nuance.com</a>>:<br>
<br>
> Any insight on what command I need to fix this? It?s the only error I have<br>
> when running gssinstallcheck.<br>
><br>
><br>
><br>
> [ERROR] Network adapter<br>
> <a href="https://www.ibm.com/support/knowledgecenter/SSYSP8_5.3.2/ess_qdg.pdf" target="_blank">
https://www.ibm.com/support/knowledgecenter/SSYSP8_5.3.2/ess_qdg.pdf</a><br>
> firmware: found 12.23.1020 expected 12.23.8010, net adapter count: 4<br>
><br>
><br>
><br>
><br>
><br>
> Bob Oesterlin<br>
><br>
> Sr Principal Storage Engineer, Nuance<br>
><br>
> 507-269-0413<br>
><br>
><br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/fad4ff57/attachment-0001.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/fad4ff57/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Wed, 3 Apr 2019 18:54:00 +0000<br>
From: "Stephen R Buchanan" <<a href="mailto:stephen.buchanan@us.ibm.com" target="_blank">stephen.buchanan@us.ibm.com</a>><br>
To: <a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>
Subject: Re: [gpfsug-discuss] New ESS install - Network adapter down<br>
level<br>
Message-ID:<br>
<<a href="mailto:OFBD2A098D.0085093E-ON002583D1.0066D1E2-002583D1.0067D25D@notes.na.collabserv.com" target="_blank">OFBD2A098D.0085093E-ON002583D1.0066D1E2-002583D1.0067D25D@notes.na.collabserv.com</a>><br>
<br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/09e229d1/attachment.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20190403/09e229d1/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
End of gpfsug-discuss Digest, Vol 87, Issue 4<br>
*********************************************<br>
</div>
</span></font></div>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div></div>