<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Helvetica, sans-serif;">
<div>
<div>My experience is that in small clusters, it’s acceptable to double up on some of these services, but as you scale up, breaking them out makes more sense. Spectrum Scale make it easy to add/remove the nodes non-disruptively, so you can move them to dedicated
nodes. When I first started testing 4.2, I setup a 6 node cluster that had both NSD and CES on them, and it did just fine. The nodes were 4-core 32GB and I had NFS and Object running on the CES nodes. The new CES nodes run a ton more services, especially when
you start working with Object.</div>
<div><br>
</div>
<div>Both of you points are valid considerations – especially with CNFS. I’m running multiple CNFS clusters and having them broken out has save me a number of times. </div>
<div>
<div id="">
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: Calibri; font-size: medium;"><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica, sans-serif; font-size: 14px;">
<font face="Helvetica">Bob Oesterlin<br>
Sr Storage Engineer, Nuance HPC Grid<br>
<br>
</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
</div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of Daniel Kidger <<a href="mailto:daniel.kidger@uk.ibm.com">daniel.kidger@uk.ibm.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>
<span style="font-weight:bold">Date: </span>Monday, January 11, 2016 at 8:42 AM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [gpfsug-discuss] GPFS 4.2 / Protocol nodes<br>
</div>
<div><br>
</div>
<div>
<div>
<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt">
<div dir="ltr">It looks like no one has attempted to answer this, so I will step in to start the conversation.</div>
<div dir="ltr"> </div>
<div dir="ltr">There are two issues when considering how many services to run on the same nodes - in this case the NSD servers.</div>
<div dir="ltr"> </div>
<div dir="ltr">1. Performance.</div>
<div dir="ltr">Spectrum Scale's (nee GPFS) core differentiator is performance. The more you run on a node the more that node resource's have to be shared. Here memory bandwidth and memory space are the main ones. CPU may also be a limited resource although
with modern chips this is less likely so.</div>
<div dir="ltr">If performance is not the key delivered metric then running other things on the NSD server may be a good option so save both cost and server spawl in small datacentres.</div>
<div dir="ltr"> </div>
<div dir="ltr">2. NFS server stability.</div>
<div dir="ltr">pre-4.1.1, IBM used cNFS to provide multiple NFS servers in a GPFS cluster. This used traditional kernel based NFS daemons. If one hung then the whole node had to be rebooted which might have led to disruption in NSD serving if the other NSD
server of a pair was already under load. With 4.1.1 came Cluster Export Services (CES) deliverd from 'Protocol Nodes'. Since there use Ganesha there would be no need to reboot this node if the NFS serving hung and in Ganesha, all NFS activity is in userspace
not the kernel.</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
</div>
</div>
</div>
</span>
</body>
</html>