<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Arial;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Helvetica Neue";
panose-1:2 0 5 3 0 0 0 2 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Excellent info Aaron. Thanks for this. I may reach out via PM to explore this more with you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span>
</b><span style="font-family:Calibri;color:black"><gpfsug-discuss-bounces@spectrumscale.org> on behalf of Aaron S Palazzolo <aspalazz@us.ibm.com><br>
<b>Reply-To: </b>gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Date: </b>Tuesday, October 25, 2016 at 2:59 PM<br>
<b>To: </b>"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
<b>Subject: </b>Re: [gpfsug-discuss] Virtualized Spectrum Scale<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">Hi Mark,</span></strong><span style="font-size:10.5pt;font-family:"Helvetica Neue""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">Great questions on the VM side. Within our Spectrum Scale Test labs at IBM, we run within both VMware and KVM. It sounds as though your questions are more towards the VMware
side so I'll give a few pointers. For more detailed info, feel free to grab me on the side or continue the discussion within the user group so that others can learn from this as well. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">#1)
</span></strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">As Luis suggests, using vmdks will not give you full support of the scsi protocol. Specifically persistent reserve. EIO errors are also not passed correctly in some path down situations.
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">- For certain test/dev environments, in which data protection and performance are not high priorities, then this may be fine. Some of our test environments do run like this. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">- Note that vmdks can be shared among virtual Spectrum Scale nodes using the multi-writer flag in VMware. To do this, you'll first need to create your vmdks using 'Thick Provision
Eager Zeroed'. You'll then need to configure the multi-writer flag for scsi-sharing such as this, for each vmdk:<o:p></o:p></span></p>
</div>
<div style="margin-left:30.0pt">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""><br>
scsi1:0.sharing = "multi-writer"<br>
scsi1:1.sharing = "multi-writer"<br>
scsi1:2.sharing = "multi-writer"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">You'll find this in the Advanced configuration parameters. Finally, you'll need to set all of these vmdks to a separate scsi adapter which is configured for either virtual or
physical sharing. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">Downsides are lack of support, some degradation of performance, lack of failover support, and inability to use VMware snapshots of VMs with scsi sharing/multi-writer enabled. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">Upsides are ease of setup both with virtually and physically and ability to store all VMs on a single datastore that can itself be snapshotted if the underlying physical storage
supports this. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">In our testlabs, we create 4node -> 50node virtual Spectrum Scale clusters, each node is a VM, some of these VMs have extra vmdks for NSDs and some do not. All VMs belonging to
a cluster reside on a single datastore which ends up being a single XIV Volume. We then can snapshot this XIV volume and in essence, snapshot the entire Spectrum Scale cluster back and forth in time. I'm sure this is overly complicated for what you want
to do, but it may get you thinking of use cases. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">#2) </span></strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">RDM will give you both performance and the piece of mind that you're using a fully supported
config. Officially, I will always recommend RDM due to this. The downside to RDM is complexity in setup unless you have a fairly static config or unless you can automate the zoning.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">#3) </span></strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">No matter what type of VM infrastructure you use, make sure you investigate the memory/cpu
requirements.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">- Check our FAQ, specifically 6.1 for tuning info regarding vm.min_free_kbytes: <a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html#gpfsclustersfaqAugust2016-gen4__lintunq">https://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html#gpfsclustersfaqAugust2016-gen4__lintunq</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">- We have run some of our test clusters with as little as 4GB of memory but I would definitely recommend quite a bit more memory in each VM for production use. If you use additional
functions other than core file system, pay attention to the memory requirements of these. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">#4) </span></strong><span style="font-size:10.5pt;font-family:"Helvetica Neue"">KVM is a viable alternative if needed..... <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""><br>
Regards,<br>
<br>
Aaron Palazzolo<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial">IBM Spectrum Scale Deployment, Infrastructure, Virtualization<br>
9042 S Rita Road, Tucson AZ 85744<br>
Phone: 520-799-5161, T/L: 321-5161<br>
E-mail: aspalazz@us.ibm.com<o:p></o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #AAAAAA 1.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue"">----- Original message -----<br>
From: gpfsug-discuss-request@spectrumscale.org<br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
To: gpfsug-discuss@spectrumscale.org<br>
Cc:<br>
Subject: gpfsug-discuss Digest, Vol 57, Issue 65<br>
Date: Tue, Oct 25, 2016 12:05 PM<br>
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Send gpfsug-discuss mailing list submissions to<br>
gpfsug-discuss@spectrumscale.org<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
or, via email, send a message with subject or body 'help' to<br>
gpfsug-discuss-request@spectrumscale.org<br>
<br>
You can reach the person managing the list at<br>
gpfsug-discuss-owner@spectrumscale.org<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of gpfsug-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. October meet the devs report<br>
(Simon Thompson (Research Computing - IT Services))<br>
2. Virtualized Spectrum Scale (Mark.Bush@siriuscom.com)<br>
3. Re: Virtualized Spectrum Scale (Luis Bolinches)<br>
4. Re: Virtualized Spectrum Scale (Kevin D Johnson)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Tue, 25 Oct 2016 15:33:16 +0000<br>
From: "Simon Thompson (Research Computing - IT Services)"<br>
<S.J.Thompson@bham.ac.uk><br>
To: "gpfsug-discuss@spectrumscale.org"<br>
<gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] October meet the devs report<br>
Message-ID:<br>
<CF45EE16DEF2FE4B9AA7FF2B6EE26545F585D875@EX13.adf.bham.ac.uk><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
<br>
The October meet the devs workshop on cloud was last week. Thanks to Dean Hildebrand and John Lewars for flying in from the US to support us, Ulf Troppens and Dan Kidger also from IBM. And finally thanks to OCF for buying the pizza (one day we'll manage to
get the pizza co to deliver on time!).<br>
<br>
The event report is now up on the group website at:<br>
<br>
<a href="http://www.spectrumscale.org/meet-the-devs-cloud-workshop-birmingham-uk" target="_blank">http://www.spectrumscale.org/meet-the-devs-cloud-workshop-birmingham-uk</a>/<br>
<br>
Simon<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Tue, 25 Oct 2016 18:46:26 +0000<br>
From: "Mark.Bush@siriuscom.com" <Mark.Bush@siriuscom.com><br>
To: "gpfsug-discuss@spectrumscale.org"<br>
<gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] Virtualized Spectrum Scale<br>
Message-ID: <1C767B09-3E62-4A4D-814C-FCFC7321A8FD@siriuscom.com><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Anyone running SpectrumScale on Virtual Machines (intel)? I?m curious how you manage disks? Do you use RDM?s? Does this even make sense to do? If you have a 2-3 node cluster how do you share the disks across? Do you have VM?s with their own VMDK?s (if
not RDM) in each node or is there some way to share access to the same VMDK?s? What are the advantages doing this other than existing HW use? Seems to me for a lab environment or very small nonperformance focused implementation this may be a viable option.<br>
<br>
Thanks<br>
<br>
Mark<br>
<br>
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law.
If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. This message may be viewed by parties at Sirius Computer Solutions other than those named in
the message header. This message does not contain an official representation of Sirius Computer Solutions. If you have received this communication in error, notify Sirius Computer Solutions immediately and (i) destroy this message if a facsimile or (ii) delete
this message immediately if this is an electronic communication. Thank you.<br>
<br>
Sirius Computer Solutions<<a href="http://www.siriuscom.com" target="_blank">http://www.siriuscom.com</a>><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/5d6808cd/attachment-0001.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/5d6808cd/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Tue, 25 Oct 2016 18:58:21 +0000<br>
From: "Luis Bolinches" <luis.bolinches@fi.ibm.com><br>
To: "gpfsug main discussion list" <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] Virtualized Spectrum Scale<br>
Message-ID:<br>
<OFDEDE10B2.7153242C-ON00258057.0068383A-1477421901449@notes.na.collabserv.com><br>
<br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi<br>
<br>
You must use RDM. Otherwise is not supported. SCSI commands is the reason.<br>
<br>
Furthermore on some versions I managed to crash the ESXi as well.<br>
<br>
--<br>
Cheers<br>
<br>
> On 25 Oct 2016, at 19.46, "Mark.Bush@siriuscom.com" <Mark.Bush@siriuscom.com> wrote:<br>
><br>
> Anyone running SpectrumScale on Virtual Machines (intel)? I?m curious how you manage disks? Do you use RDM?s? Does this even make sense to do? If you have a 2-3 node cluster how do you share the disks across? Do you have VM?s with their own VMDK?s (if
not RDM) in each node or is there some way to share access to the same VMDK?s? What are the advantages doing this other than existing HW use? Seems to me for a lab environment or very small nonperformance focused implementation this may be a viable option.<br>
> <br>
> Thanks<br>
> <br>
> Mark<br>
> This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law.
If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. This message may be viewed by parties at Sirius Computer Solutions other than those named in
the message header. This message does not contain an official representation of Sirius Computer Solutions. If you have received this communication in error, notify Sirius Computer Solutions immediately and (i) destroy this message if a facsimile or (ii) delete
this message immediately if this is an electronic communication. Thank you.<br>
><br>
> Sirius Computer Solutions<br>
<br>
Ellei edell? ole toisin mainittu: / Unless stated otherwise above:<br>
Oy IBM Finland Ab<br>
PL 265, 00101 Helsinki, Finland<br>
Business ID, Y-tunnus: 0195876-3<br>
Registered in Finland<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/68a76094/attachment-0001.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/68a76094/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Tue, 25 Oct 2016 19:05:12 +0000<br>
From: "Kevin D Johnson" <kevindjo@us.ibm.com><br>
To: gpfsug-discuss@spectrumscale.org<br>
Cc: gpfsug-discuss@spectrumscale.org<br>
Subject: Re: [gpfsug-discuss] Virtualized Spectrum Scale<br>
Message-ID:<br>
<OF058EA313.ABD3EA09-ON00258057.0068A1BE-00258057.0068D8AA@notes.na.collabserv.com><br>
<br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/a0853f55/attachment.html" target="_blank">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/a0853f55/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
End of gpfsug-discuss Digest, Vol 57, Issue 65<br>
**********************************************</span><span style="font-size:10.5pt;font-family:"Helvetica Neue""><br>
<o:p></o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Helvetica Neue""> <o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>