<div class="socmaildefaultfont" dir="ltr" style="font-family:"Helvetica Neue", Helvetica, Arial, sans-serif;font-size:10.5pt" ><div dir="ltr" ><strong>Hi Mark,</strong></div>
<div dir="ltr" > </div>
<div dir="ltr" >Great questions on the VM side. Within our Spectrum Scale Test labs at IBM, we run within both VMware and KVM. It sounds as though your questions are more towards the VMware side so I'll give a few pointers. For more detailed info, feel free to grab me on the side or continue the discussion within the user group so that others can learn from this as well. </div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" ><strong>#1) </strong>As Luis suggests, using vmdks will not give you full support of the scsi protocol. Specifically persistent reserve. EIO errors are also not passed correctly in some path down situations. </div>
<div dir="ltr" >- For certain test/dev environments, in which data protection and performance are not high priorities, then this may be fine. Some of our test environments do run like this. </div>
<div dir="ltr" >- Note that vmdks can be shared among virtual Spectrum Scale nodes using the multi-writer flag in VMware. To do this, you'll first need to create your vmdks using 'Thick Provision Eager Zeroed'. You'll then need to configure the multi-writer flag for scsi-sharing such as this, for each vmdk:</div>
<div dir="ltr" style="margin-left: 40px;" ><br>scsi1:0.sharing = "multi-writer"<br>scsi1:1.sharing = "multi-writer"<br>scsi1:2.sharing = "multi-writer"</div>
<div dir="ltr" > </div>
<div dir="ltr" >You'll find this in the Advanced configuration parameters. Finally, you'll need to set all of these vmdks to a separate scsi adapter which is configured for either virtual or physical sharing. </div>
<div dir="ltr" > </div>
<div dir="ltr" >Downsides are lack of support, some degradation of performance, lack of failover support, and inability to use VMware snapshots of VMs with scsi sharing/multi-writer enabled. </div>
<div dir="ltr" >Upsides are ease of setup both with virtually and physically and ability to store all VMs on a single datastore that can itself be snapshotted if the underlying physical storage supports this. </div>
<div dir="ltr" > </div>
<div dir="ltr" >In our testlabs, we create 4node -> 50node virtual Spectrum Scale clusters, each node is a VM, some of these VMs have extra vmdks for NSDs and some do not. All VMs belonging to a cluster reside on a single datastore which ends up being a single XIV Volume. We then can snapshot this XIV volume and in essence, snapshot the entire Spectrum Scale cluster back and forth in time. I'm sure this is overly complicated for what you want to do, but it may get you thinking of use cases. </div>
<div dir="ltr" > </div>
<div dir="ltr" ><strong>#2) </strong>RDM will give you both performance and the piece of mind that you're using a fully supported config. Officially, I will always recommend RDM due to this. The downside to RDM is complexity in setup unless you have a fairly static config or unless you can automate the zoning.</div>
<div dir="ltr" > </div>
<div dir="ltr" ><strong>#3) </strong>No matter what type of VM infrastructure you use, make sure you investigate the memory/cpu requirements.</div>
<div dir="ltr" >- Check our FAQ, specifically 6.1 for tuning info regarding vm.min_free_kbytes: <a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html#gpfsclustersfaqAugust2016-gen4__lintunq" >https://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html#gpfsclustersfaqAugust2016-gen4__lintunq</a></div>
<div dir="ltr" >- We have run some of our test clusters with as little as 4GB of memory but I would definitely recommend quite a bit more memory in each VM for production use. If you use additional functions other than core file system, pay attention to the memory requirements of these. </div>
<div dir="ltr" > </div>
<div dir="ltr" ><strong>#4) </strong>KVM is a viable alternative if needed..... </div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" ><br>Regards,<br><br>Aaron Palazzolo</div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div dir="ltr" >IBM Spectrum Scale Deployment, Infrastructure, Virtualization<br>9042 S Rita Road, Tucson AZ 85744<br>Phone: 520-799-5161, T/L: 321-5161<br>E-mail: aspalazz@us.ibm.com</div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: gpfsug-discuss-request@spectrumscale.org<br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug-discuss@spectrumscale.org<br>Cc:<br>Subject: gpfsug-discuss Digest, Vol 57, Issue 65<br>Date: Tue, Oct 25, 2016 12:05 PM<br>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >Send gpfsug-discuss mailing list submissions to<br>gpfsug-discuss@spectrumscale.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>or, via email, send a message with subject or body 'help' to<br>gpfsug-discuss-request@spectrumscale.org<br><br>You can reach the person managing the list at<br>gpfsug-discuss-owner@spectrumscale.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of gpfsug-discuss digest..."<br><br><br>Today's Topics:<br><br> 1. October meet the devs report<br> (Simon Thompson (Research Computing - IT Services))<br> 2. Virtualized Spectrum Scale (Mark.Bush@siriuscom.com)<br> 3. Re: Virtualized Spectrum Scale (Luis Bolinches)<br> 4. Re: Virtualized Spectrum Scale (Kevin D Johnson)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Tue, 25 Oct 2016 15:33:16 +0000<br>From: "Simon Thompson (Research Computing - IT Services)"<br><S.J.Thompson@bham.ac.uk><br>To: "gpfsug-discuss@spectrumscale.org"<br><gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] October meet the devs report<br>Message-ID:<br><CF45EE16DEF2FE4B9AA7FF2B6EE26545F585D875@EX13.adf.bham.ac.uk><br>Content-Type: text/plain; charset="us-ascii"<br><br><br>The October meet the devs workshop on cloud was last week. Thanks to Dean Hildebrand and John Lewars for flying in from the US to support us, Ulf Troppens and Dan Kidger also from IBM. And finally thanks to OCF for buying the pizza (one day we'll manage to get the pizza co to deliver on time!).<br><br>The event report is now up on the group website at:<br><br><a href="http://www.spectrumscale.org/meet-the-devs-cloud-workshop-birmingham-uk" target="_blank" >http://www.spectrumscale.org/meet-the-devs-cloud-workshop-birmingham-uk</a>/<br><br>Simon<br><br>------------------------------<br><br>Message: 2<br>Date: Tue, 25 Oct 2016 18:46:26 +0000<br>From: "Mark.Bush@siriuscom.com" <Mark.Bush@siriuscom.com><br>To: "gpfsug-discuss@spectrumscale.org"<br><gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] Virtualized Spectrum Scale<br>Message-ID: <1C767B09-3E62-4A4D-814C-FCFC7321A8FD@siriuscom.com><br>Content-Type: text/plain; charset="utf-8"<br><br>Anyone running SpectrumScale on Virtual Machines (intel)? I?m curious how you manage disks? Do you use RDM?s? Does this even make sense to do? If you have a 2-3 node cluster how do you share the disks across? Do you have VM?s with their own VMDK?s (if not RDM) in each node or is there some way to share access to the same VMDK?s? What are the advantages doing this other than existing HW use? Seems to me for a lab environment or very small nonperformance focused implementation this may be a viable option.<br><br>Thanks<br><br>Mark<br><br>This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. This message may be viewed by parties at Sirius Computer Solutions other than those named in the message header. This message does not contain an official representation of Sirius Computer Solutions. If you have received this communication in error, notify Sirius Computer Solutions immediately and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.<br><br>Sirius Computer Solutions<<a href="http://www.siriuscom.com" target="_blank" >http://www.siriuscom.com</a>><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/5d6808cd/attachment-0001.html" target="_blank" >http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/5d6808cd/attachment-0001.html</a>><br><br>------------------------------<br><br>Message: 3<br>Date: Tue, 25 Oct 2016 18:58:21 +0000<br>From: "Luis Bolinches" <luis.bolinches@fi.ibm.com><br>To: "gpfsug main discussion list" <gpfsug-discuss@spectrumscale.org><br>Subject: Re: [gpfsug-discuss] Virtualized Spectrum Scale<br>Message-ID:<br><OFDEDE10B2.7153242C-ON00258057.0068383A-1477421901449@notes.na.collabserv.com><br><br>Content-Type: text/plain; charset="utf-8"<br><br>Hi<br><br>You must use RDM. Otherwise is not supported. SCSI commands is the reason.<br><br>Furthermore on some versions I managed to crash the ESXi as well.<br><br>--<br>Cheers<br><br>> On 25 Oct 2016, at 19.46, "Mark.Bush@siriuscom.com" <Mark.Bush@siriuscom.com> wrote:<br>><br>> Anyone running SpectrumScale on Virtual Machines (intel)? I?m curious how you manage disks? Do you use RDM?s? Does this even make sense to do? If you have a 2-3 node cluster how do you share the disks across? Do you have VM?s with their own VMDK?s (if not RDM) in each node or is there some way to share access to the same VMDK?s? What are the advantages doing this other than existing HW use? Seems to me for a lab environment or very small nonperformance focused implementation this may be a viable option.<br>> <br>> Thanks<br>> <br>> Mark<br>> This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. This message may be viewed by parties at Sirius Computer Solutions other than those named in the message header. This message does not contain an official representation of Sirius Computer Solutions. If you have received this communication in error, notify Sirius Computer Solutions immediately and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.<br>><br>> Sirius Computer Solutions<br><br>Ellei edell? ole toisin mainittu: / Unless stated otherwise above:<br>Oy IBM Finland Ab<br>PL 265, 00101 Helsinki, Finland<br>Business ID, Y-tunnus: 0195876-3<br>Registered in Finland<br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/68a76094/attachment-0001.html" target="_blank" >http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/68a76094/attachment-0001.html</a>><br><br>------------------------------<br><br>Message: 4<br>Date: Tue, 25 Oct 2016 19:05:12 +0000<br>From: "Kevin D Johnson" <kevindjo@us.ibm.com><br>To: gpfsug-discuss@spectrumscale.org<br>Cc: gpfsug-discuss@spectrumscale.org<br>Subject: Re: [gpfsug-discuss] Virtualized Spectrum Scale<br>Message-ID:<br><OF058EA313.ABD3EA09-ON00258057.0068A1BE-00258057.0068D8AA@notes.na.collabserv.com><br><br>Content-Type: text/plain; charset="us-ascii"<br><br>An HTML attachment was scrubbed...<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/a0853f55/attachment.html" target="_blank" >http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161025/a0853f55/attachment.html</a>><br><br>------------------------------<br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br><br><br>End of gpfsug-discuss Digest, Vol 57, Issue 65<br>**********************************************</font><br> </div></blockquote>
<div dir="ltr" > </div></div><BR>