<div dir="ltr"><div>I agree with Luis -- why so many nodes?<br><br>"""<br><div>So if i would go for 4 NSD server, 6 protocol nodes and 2 tsm
backup nodes and at least 3 test server a total of 11 server is needed.</div><div>"""<br><br></div><div>If this is your whole cluster, why not just 3x P822L/P812L running single partition per node, hosting a cluster of 3x protocol-nodes that does both direct FC for disk access, and also run backups on same nodes ? No complications, full hw performance. Then separate node for test, or separate partition on same nodes with dedicated adapters.<br><br></div><div>But back to your original question. My experience is that LPAR/NPIV works great, but it's a bit annoying having to also have VIOs. Hope we'll get FC SR-IOV eventually.. Also LPAR/Dedicated-adapters naturally works fine.<br><br>VMWare/RDM can be a challenge in some failure situations. It likes to pause VMs in APD or PDL situations, which will affect all VMs with access to it :-o<br><br></div><div>VMs without direct disk access is trivial.<br></div><br><br><br></div> -jf<br><div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 24, 2017 at 2:42 PM, Luis Bolinches <span dir="ltr"><<a href="mailto:luis.bolinches@fi.ibm.com" target="_blank">luis.bolinches@fi.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font size="2" face="sans-serif">Hi</font>
<br>
<br><font size="2" face="sans-serif">As tastes vary, I would not partition
it so much for the backend. Assuming there is little to nothing overhead
on the CPU at PHYP level, which it depends. On the protocols nodes, due
the CTDB keeping locks together across all nodes (SMB), you would get more
performance on bigger & less number of CES nodes than more and smaller.</font>
<br>
<br><font size="2" face="sans-serif">Certainly a 822 is quite a server if
we go back to previous generations but I would still keep a simple backend
(NSd servers), simple CES (less number of nodes the merrier) & then
on the client part go as micro partitions as you like/can as the effect
on the cluster is less relevant in the case of resources starvation.</font>
<br>
<br><font size="2" face="sans-serif">But, it depends on workloads, SLA and
money so I say try, establish a baseline and it fills the requirements,
go for it. If not change till does. Have fun</font>
<br>
<br>
<br>
<br><font size="1" face="sans-serif" color="#5f5f5f">From:
</font><font size="1" face="sans-serif">"<a href="mailto:service@metamodul.com" target="_blank">service@metamodul.com</a>"
<<a href="mailto:service@metamodul.com" target="_blank">service@metamodul.com</a>></font>
<br><font size="1" face="sans-serif" color="#5f5f5f">To:
</font><font size="1" face="sans-serif">gpfsug main discussion
list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.<wbr>org</a>></font>
<br><font size="1" face="sans-serif" color="#5f5f5f">Date:
</font><font size="1" face="sans-serif">24/04/2017 15:21</font>
<br><font size="1" face="sans-serif" color="#5f5f5f">Subject:
</font><font size="1" face="sans-serif">Re: [gpfsug-discuss]
Used virtualization technologies for GPFS/Spectrum Scale</font>
<br><font size="1" face="sans-serif" color="#5f5f5f">Sent by:
</font><font size="1" face="sans-serif"><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@<wbr>spectrumscale.org</a></font>
<br>
<hr noshade><div><div class="h5">
<br>
<br>
<br><font size="3">Hi Jonathan</font>
<br><font size="3">todays hardware is so powerful that imho it might make
sense to split a CEC into more "piece". For example the IBM S822L
has up to 2x12 cores, 9 PCI3 slots ( 4×16 lans & 5×8 lan ).</font>
<br><font size="3">I think that such a server is a little bit to big just
to be a single NSD server.</font>
<br><font size="3">Note that i use for each GPFS service a dedicated node.</font>
<br><font size="3">So if i would go for 4 NSD server, 6 protocol nodes and
2 tsm backup nodes and at least 3 test server a total of 11 server is needed.</font>
<br><font size="3">Inhm 4xS822L could handle this and a little bit more quite
well.</font>
<br>
<br><font size="3">Of course blade technology could be used or 1U server.</font>
<br>
<br><font size="3">With kind regards</font>
<br><font size="3">Hajo</font>
<br>
<br><font size="3">-- </font>
<br><font size="3">Unix Systems Engineer</font>
<br><font size="3">MetaModul GmbH</font>
<br><font size="3"><a href="tel:+49%20177%204393994" value="+491774393994" target="_blank">+49 177 4393994</a></font>
<br><font size="3"><br>
</font>
<br><font size="3">-------- Ursprüngliche Nachricht --------</font>
<br><font size="3">Von: Jonathan Buzzard </font>
<br><font size="3">Datum:2017.04.24 13:14 (GMT+01:00) </font>
<br><font size="3">An: gpfsug main discussion list </font>
<br><font size="3">Betreff: Re: [gpfsug-discuss] Used virtualization technologies
for GPFS/Spectrum Scale </font>
<br>
<br><font size="3">On Mon, 2017-04-24 at 12:28 +0200, Hans-Joachim Ehlers
wrote:<br>
> @All<br>
> <br>
> <br>
> does anybody uses virtualization technologies for GPFS Server ? If
yes<br>
> what kind and why have you selected your soulution.<br>
> <br>
> I think currently about using Linux on Power using 40G SR-IOV for<br>
> Network and NPIV/Dedidcated FC Adater for storage. As a plus i can<br>
> also assign only a certain amount of CPUs to GPFS. ( Lower license<br>
> cost / You pay for what you use)<br>
> <br>
> <br>
> I must admit that i am not familar how "good" KVM/ESX in
respect to<br>
> direct assignment of hardware is. Thus the question to the group<br>
> <br>
<br>
For the most part GPFS is used at scale and in general all the<br>
components are redundant. As such why you would want to allocate less<br>
than a whole server into a production GPFS system in somewhat beyond me.<br>
<br>
That is you will have a bunch of NSD servers in the system and if one<br>
crashes, well the other NSD's take over. Similar for protocol nodes, and<br>
in general the total file system size is going to hundreds of TB<br>
otherwise why bother with GPFS.<br>
<br>
I guess there is currently potential value at sticking the GUI into a<br>
virtual machine to get redundancy.<br>
<br>
On the other hand if you want a test rig, then virtualization works<br>
wonders. I have put GPFS on a single Linux box, using LV's for the disks<br>
and mapping them into virtual machines under KVM.<br>
<br>
JAB.<br>
<br>
-- <br>
Jonathan A. Buzzard
Email: jonathan (at) <a href="http://buzzard.me.uk" target="_blank">buzzard.me.uk</a><br>
Fife, United Kingdom.<br>
<br>
______________________________<wbr>_________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
</font></div></div><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><font size="3">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</font></a><tt><font size="2">_______<wbr>______________________________<wbr>__________<span class=""><br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
</span></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br>
</font></tt>
<br>
<br><font size="2" face="sans-serif"><br>
<br>
Ellei edellä ole toisin mainittu: / Unless stated otherwise above:<br>
Oy IBM Finland Ab<br>
PL 265, 00101 Helsinki, Finland<br>
Business ID, Y-tunnus: 0195876-3 <br>
Registered in Finland<br>
</font><br>______________________________<wbr>_________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br>
<br></blockquote></div><br></div>