<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:9pt" ><div dir="ltr" >> In fact, one of the things that’s kinda surprising to me is that upgrading the SMB portion of CES requires a downtime. Let’s just say that I know for a fact that sernet-samba can be done rolling / live.</div>
<div dir="ltr" > </div>
<div dir="ltr" >I am referring to the open source version of Samba here. That is likely close to "sernet-samba", but i have not seen the details on the code they use for the package build:</div>
<div dir="ltr" > </div>
<div dir="ltr" >Clustered Samba with ctdb never supported rolling code upgrades. The reason for this is that SMB-level records are shared across the protocol nodes through ctdb. These records are not versioned and Samba on each node expects to only see matching records. As the details of the internal data shared across the nodes can change through versions, the only safe way to handle this is to not allow rolling code upgrades.</div>
<div dir="ltr" > </div>
<div dir="ltr" >It might have appeared that Samba supports rolling code upgrades. Past versions did not check for version compatibility across the nodes, so there was no warning. If the ctdb records shared between the nodes did not change, then this would be no problem (for this particular upgrade path, it is likely different for different Samba versions). Also, if there are no open files or active sessions during the upgrade, the risk is lower, as in that case there are fewer records that could cause problems.</div>
<div dir="ltr" > </div>
<div dir="ltr" >The important change is that Samba 4.7.0 introduced checks to enforce compatible versions across the nodes. This just makes the limitation visible, but it was always there. See:</div>
<div dir="ltr" ><a href="https://www.samba.org/samba/history/samba-4.7.0.html" >https://www.samba.org/samba/history/samba-4.7.0.html</a></div>
<div dir="ltr" ><pre>* CTDB no longer allows mixed minor versions in a cluster
See the AllowMixedVersions tunable option in ctdb-tunables(7) and also
<a href="https://wiki.samba.org/index.php/Upgrading_a_CTDB_cluster#Policy" >https://wiki.samba.org/index.php/Upgrading_a_CTDB_cluster#Policy</a></pre></div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div dir="ltr" > </div>
<div dir="ltr" >and also</div>
<div dir="ltr" ><a href="https://wiki.samba.org/index.php/Upgrading_a_CTDB_cluster" >https://wiki.samba.org/index.php/Upgrading_a_CTDB_cluster</a></div>
<div dir="ltr" >"<span class="mw-headline" id="Rolling_Upgrades" >Rolling Upgrades" and "Problems with Rolling Code Upgrades"</span></div>
<div dir="ltr" >Note that this page refers to two different layers. "ctdb" itself maintains compatibility among a X.Y. code stream, but this is not guaranteed for the file server records stored in ctdb databases.</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" >We have the same limitation and enforcement in the Samba version shipped with Spectrum Scale. I expect the same to be true for all clustered Samba versions today.</div>
<div dir="ltr" > </div>
<div dir="ltr" >Regards,</div>
<div dir="ltr" ><br><font size="2" face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" ><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" >Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ<br>christof.schmitt@us.ibm.com || +1-520-799-2469 (T/L: 321-2469)</font></font></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Buterbaugh, Kevin L" <Kevin.Buterbaugh@Vanderbilt.Edu><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: Re: [gpfsug-discuss] Not recommended, but why not?<br>Date: Fri, May 4, 2018 12:12 PM<br> <br><!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >--> Hi Anderson,
<div> </div>
<div>Thanks for the response … however, the scenario you describe below wouldn’t impact us. We have 8 NSD servers and they can easily provide the needed performance to native GPFS clients. We could also take a downtime if we ever did need to expand in the manner described below.</div>
<div> </div>
<div>In fact, one of the things that’s kinda surprising to me is that upgrading the SMB portion of CES requires a downtime. Let’s just say that I know for a fact that sernet-samba can be done rolling / live.</div>
<div> </div>
<div>Kevin
<div>
<blockquote type="cite" ><div>On May 4, 2018, at 10:52 AM, Anderson Ferreira Nobre <<a href="mailto:anobre@br.ibm.com" target="_blank" >anobre@br.ibm.com</a>> wrote:</div>
<div><div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >Hi Kevin,</div>
<div dir="ltr" > </div>
<div dir="ltr" >I think one of the reasons is if you need to add or remove nodes from cluster you will start to face the constrains of this kind of solution. Let's say you have a cluster with two nodes and share the same set of LUNs through SAN. And for some reason you need to add more two nodes that are NSD Servers and Protocol nodes. For the new nodes become NSD Servers, you will have to redistribute the NSD disks among four nodes. But for you do that you will have to umount the filesystems. And for you umount the filesystems you would need to stop protocol services. At the end you will realize that a simple task like that is disrruptive. You won't be able to do online.</div>
<div dir="ltr" > </div>
<div dir="ltr" ><div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" ><div> </div>
<table style="border:0;table-layout:auto;white-space:normal;color:#5f5f5f;width:650px;font-size:7.2pt" cellspacing="0" cellpadding="0" width="650px" > <tbody> <tr> <td style="vertical-align: bottom;" > <div style="font-size:9pt" ><span style="font-size:1.000em;" >Abraços / Regards / Saludos,</span></div>
<div> </div>
<div style="margin:0 0 8px 0" ><span style="font-size:1.000em;" ><b style="color:#888888;font-size:12pt" ><span>Anderson</span> <span>Nobre</span></b><br> AIX & Power Consultant<br> Master Certified IT Specialist<br> IBM Systems Hardware Client Technical Team – IBM Systems Lab Services<br> <br> <img alt="community_general_lab_services" s_org_src="http://esign.webmaster.ibm.com/img/signature_lab_services.jpg" width="333" height="120" ></span></div> </td> <td> </td> </tr> </tbody></table>
<table style="border:0;table-layout:auto;white-space:normal;color:#5f5f5f;width:650px;font-size:7.2pt" cellspacing="0" cellpadding="0" width="650px" > <tbody> <tr> <td colspan="3" > <hr style="border:0;color:#696969;background-color:#696969;height:1px;" ></td> </tr> <tr> <td style="vertical-align: top;font-family:arial,helvetica,sans-serif" width="400px" ><span style="font-size:1.000em;" ><span><span style="color:#466BB0;font-weight:bold" ><abbr title="work" >Phone:</abbr></span> <span>55-19-2132-4317</span></span><br> <span style="color:#466BB0;font-weight:bold;font-style:7.2pt" >E-mail:</span> <a href="mailto:anobre@br.ibm.com" target="_blank" ><span style="color:#5f5f5f" >anobre@br.ibm.com</span></a></span></td> <td style="vertical-align: top;font-family:arial,helvetica,sans-serif" align="right" width="150px" ><span style="font-size:1.000em;" ><img alt="IBM" s_org_src="http://esign.webmaster.ibm.com/img/ibm3.gif" ></span></td> </tr> </tbody></table></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Buterbaugh, Kevin L" <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu" target="_blank" >Kevin.Buterbaugh@Vanderbilt.Edu</a>><br>Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank" >gpfsug-discuss-bounces@spectrumscale.org</a><br>To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>><br>Cc:<br>Subject: [gpfsug-discuss] Not recommended, but why not?<br>Date: Fri, May 4, 2018 12:39 PM<br> <br><!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->Hi All,
<div> </div>
<div>In doing some research, I have come across numerous places (IBM docs, DeveloperWorks posts, etc.) where it is stated that it is not recommended to run CES on NSD servers … but I’ve not found any detailed explanation of why not.</div>
<div> </div>
<div>I understand that CES, especially if you enable SMB, can be a resource hog. But if I size the servers appropriately … say, late model boxes with 2 x 8 core CPU’s, 256 GB RAM, 10 GbE networking … is there any reason why I still should not combine the two?</div>
<div> </div>
<div>To answer the question of why I would want to … simple, server licenses.</div>
<div> </div>
<div>Thanks…</div>
<div> </div>
<div>Kevin</div>
<div><div>—</div>
<div>Kevin Buterbaugh - Senior System Administrator</div>
<div>Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" target="_blank" >Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
<div> </div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=SKGbPzt272e5SYT-aBXleEEJEr_IHcgiNwByXE7pB4s&e=" target="_blank" >spectrumscale.org</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fgpfsug.org-252Fmailman-252Flistinfo-252Fgpfsug-2Ddiscuss-26data-3D02-257C01-257CKevin.Buterbaugh-2540vanderbilt.edu-257C2b0fc12c4dc24aa1f7fb08d5b1d70c9e-257Cba5a7f39e3be4ab3b45067fa80faecad-257C0-257C1-257C636610459542553835-26sdata-3D8aArQLzU5q-252BySqHcoQ3SI420XzP08ICph7F18G7C4pw-253D-26reserved-3D0&d=DwMGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=rROtdCL7ZogTmDFbiKSiUnb-fXB1gDmH9o1k1m0nM8o&e=" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></div></blockquote>
<div dir="ltr" > </div></div><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=SKGbPzt272e5SYT-aBXleEEJEr_IHcgiNwByXE7pB4s&e=" target="_blank" >spectrumscale.org</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fgpfsug.org-252Fmailman-252Flistinfo-252Fgpfsug-2Ddiscuss-26data-3D02-257C01-257CKevin.Buterbaugh-2540vanderbilt.edu-257C2b0fc12c4dc24aa1f7fb08d5b1d70c9e-257Cba5a7f39e3be4ab3b45067fa80faecad-257C0-257C1-257C636610459542553835-26sdata-3D8aArQLzU5q-252BySqHcoQ3SI420XzP08ICph7F18G7C4pw-253D-26reserved-3D0&d=DwMGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=rROtdCL7ZogTmDFbiKSiUnb-fXB1gDmH9o1k1m0nM8o&e=" target="_blank" >https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C2b0fc12c4dc24aa1f7fb08d5b1d70c9e%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636610459542553835&sdata=8aArQLzU5q%2BySqHcoQ3SI420XzP08ICph7F18G7C4pw%3D&reserved=0</a></div></blockquote></div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=OsAi6TqFQWMLCf9mCo-kmrkLku2WGXLM8jVumnnqZBs&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=5Nn7eUPeYe291x8f39jKybESLKv_W_XtkTkS8fTR-NI&m=MTn8f-19YeL-zMa1zGhHI12qDt0L0uHnCDQ8nGsPWRs&s=OsAi6TqFQWMLCf9mCo-kmrkLku2WGXLM8jVumnnqZBs&e=</a></font></div></blockquote>
<div dir="ltr" > </div></div><BR>