<font face="Verdana,Arial,Helvetica,sans-serif" size="2"> <span>Mark<br><br>In addition to the replicas, the node quorum, and a tiebreaker disk, you need to follow recommended practice of third site with a very small node for node quorum and file system quorum. <br><br>What I think is missing here is what is often called a third site consisting of something as simple as a Laptop. You can build two node clusters with a local tiebreaker disk to avoid split brain. But there is also a need for a third failure group that holds a copy of the File System Descriptors typically, instantiated as descOnly in the NSD, to maintain file system quorum in addtion to node quorum. Often this is done at a third location so if one location goes down quorum on nodes and file system are maintained as well as a tiebreaker supported. The third site could be an NSD internal to the laptop, not on a SAN, of about 128MB in size.<br><br>There is a good discussion on Synchronous Mirroring in the Spectrum Scale advanced adminstrstion guide. In the command "mmcrnsd" the usage is descOnly and states:<br>descOnly<br>Indicates that the disk contains no data and no file metadata. Such a disk is used solely to keep a copy of the file system descriptor, and can be used as a third failure group in certain disaster recovery configurations. For more information, see the help topic "Synchronous mirroring utilizing GPFS replication" in IBM Spectrum Scale: Advanced Administration Guide. See page 85 "Establishing disaster recovery for a cluster. Ignore the discussion on PPRC and Flashcopy as these are not related to your question. <br><br>A pdf of all the manuals is at <a target="_blank" href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/ibmspectrumscale42_content.html">https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/ibmspectrumscale42_content.html</a><br><br>There is also a description of the File System Descriptor Quorum in the Concepts, Planning, and Installation Guides page 52 (chapter 2 in general is worth reading for your area of questioning). If one site was failure group 1 and the other failure group 2, then a replication of 2 would force the synchronous mirror to occur across your two sites. But you could loose quorum on the file system and therefore while the cluster would be running, the file systems may not. It states<br><br></span><br><span>
</span><div class="page" title="Page 72"> <div class="layoutArea"> <div class="column"> <p><span style="font-size: 12.000000pt; font-family: 'Helvetica'; font-weight: 700">File system descriptor quorum
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'Palatino'">A GPFS structure called the </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-style: italic">file system descriptor </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">is initially written to every disk in the file system and is
replicated on a subset of the disks as changes to the file system occur, such as the adding or deleting of
disks. Based on the number of failure groups and disks, GPFS creates one to five replicas of the
descriptor:
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are at least five different failure groups, five replicas are created. </span><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'"><br></span></p><p><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'"></span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are at least three different disks, three replicas are created.<br></span><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are only one or two disks, a replica is created on each disk.
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'Palatino'"><br></span></p><p><span style="font-size: 10.000000pt; font-family: 'Palatino'">Once it decides how many replicas to create, GPFS picks disks to hold the replicas, so that all replicas are
in different failure groups, if possible, to reduce the risk of multiple failures. In picking replica locations,
the current state of the disks is taken into account. Stopped or suspended disks are avoided. Similarly,
when a failed disk is brought back online, GPFS might rebalance the file system descriptors in order to
assure reliability across the failure groups. The disks used to hold the file system descriptor replicas can
be seen by running the </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-weight: 700">mmlsdisk </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-style: italic">fsname </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-weight: 700">-L </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">command and looking for the string </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-weight: 700">desc </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">in the </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-weight: 700">Remarks
</span><span style="font-size: 10.000000pt; font-family: 'Palatino'">column.
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'Palatino'">GPFS requires that a majority of the replicas on the subset of disks remain available to sustain file system
operations:
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are at least five different replicas, GPFS can tolerate a loss of two of the five replicas.<br></span><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are at least three replicas, GPFS can tolerate a loss of one of the three replicas.<br></span><span style="font-size: 10.000000pt; font-family: 'XyvisionPiOne'">- </span><span style="font-size: 10.000000pt; font-family: 'Palatino'">If there are fewer than three replicas, a loss of one replica might make the descriptor inaccessible.
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'Palatino'"><br></span></p><p><span style="font-size: 10.000000pt; font-family: 'Palatino'"><span style="background-color: rgb(255, 255, 0);">The loss of all disks in a disk failure group might cause a majority of file systems descriptors to become
unavailable and inhibit further file system operations. </span>For example, if your file system is backed up by
three or more disks that are assigned to two separate disk failure groups, one of the failure groups will
be assigned two of the file system descriptor replicas, while the other failure group will be assigned only
one replica. If all of the disks in the disk failure group that contains the two replicas were to become
unavailable, the file system would also become unavailable. To avoid this particular scenario, you might
want to introduce a third disk failure group consisting of a single disk that is designated as a </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-weight: 700">descOnly
</span><span style="font-size: 10.000000pt; font-family: 'Palatino'">disk. This disk would exist solely to contain a replica of the file system descriptor (that is, it would not
contain any file system metadata or data). This disk should be at least 128MB in size.
</span></p> <p><span style="font-size: 10.000000pt; font-family: 'Palatino'">For more information on this topic, see “Network Shared Disk (NSD) creation considerations” on page 50
and the topic "Establishing disaster recovery for your GPFS cluster" in the </span><span style="font-size: 10.000000pt; font-family: 'Palatino'; font-style: italic">IBM Spectrum Scale: Advanced
Administration Guide</span><span style="font-size: 10.000000pt; font-family: 'Palatino'">.
</span></p> </div> </div> </div>
<span><br>Edward L. Boyd ( Ed )<br>Client Technical Specialist, Spectrum Scale (GPFS) / Spectrum Compute (Platform) / IBM Cloud Storage (Cleversafe) / Elastic Storage Server<br>IBM Systems, Software Defined Storage Solutions (SDSS)<br>US Federal<br>407-221-9544 Cell / Text Msg<br><a href="mailto:eboyd@us.ibm.com">eboyd@us.ibm.com</a> email</span><br><br><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" color="#000000" size="2"><font color="#990099"><a target="_blank" href="mailto:-----gpfsug-discuss-bounces@spectrumscale.org">-----gpfsug-discuss-bounces@spectrumscale.org</a> wrote: -----</font><div class="iNotesHistory" style="padding-left:5px;"><div style="padding-right:0px;padding-left:5px;border-left:solid black 2px;">To: <a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><br>From: <a target="_blank" href="mailto:gpfsug-discuss-request@spectrumscale.org">gpfsug-discuss-request@spectrumscale.org</a><br>Sent by: <a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><br>Date: 07/21/2016 09:13AM<br>Subject: gpfsug-discuss Digest, Vol 54, Issue 48<br><br><div><font face="Courier New,Courier,monospace" size="2">Send gpfsug-discuss mailing list submissions to<br> <a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><br><br>To subscribe or unsubscribe via the World Wide Web, visit<br> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>or, via email, send a message with subject or body 'help' to<br> <a target="_blank" href="mailto:gpfsug-discuss-request@spectrumscale.org">gpfsug-discuss-request@spectrumscale.org</a><br><br>You can reach the person managing the list at<br> <a target="_blank" href="mailto:gpfsug-discuss-owner@spectrumscale.org">gpfsug-discuss-owner@spectrumscale.org</a><br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of gpfsug-discuss digest..."<br><br><br>Today's Topics:<br><br> 1. Re: NDS in Two Site scenario (<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a>)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Thu, 21 Jul 2016 13:12:58 +0000<br>From: "<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a>" <<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a>><br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br>Message-ID: <<a target="_blank" href="mailto:10D22907-E641-41AF-A31A-17755288E005@siriuscom.com">10D22907-E641-41AF-A31A-17755288E005@siriuscom.com</a>><br>Content-Type: text/plain; charset="utf-8"<br><br>Thanks Vic&Simon, I?m totally cool with ?it depends? the solution guidance is to achieve a Highly Available FS. And there is Dark Fibre between the two locations. FileNet is the application and they want two things. Ability to write in both locations (maybe close to at the same time not necessarily the same files though) and protect against any site failure. So in my mind my Scenario 1 would work as long as I had copies=2 and restripe are acceptable. Is my Scenario 2 I would still have to restripe if the SAN in site 1 went down.<br><br>I?m looking for the simplest approach that provides the greatest availability.<br><br><br><br>From: <<a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of "Simon Thompson (Research Computing - IT Services)" <<a target="_blank" href="mailto:S.J.Thompson@bham.ac.uk">S.J.Thompson@bham.ac.uk</a>><br>Reply-To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>Date: Thursday, July 21, 2016 at 8:02 AM<br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br><br>It depends.<br><br>What are you protecting against?<br><br>Either will work depending on your acceptable failure modes. I'm assuming here that you are using copies=2 to replicate the data, and that the NSD devices have different failure groups per site.<br><br>In the second example, if you were to lose the NSD servers in Site 1, but not the SAN, you would continue to have 2 copies of data written as the NSD servers in Site 2 could write to the SAN in Site 1.<br><br>In the first example you would need to rest ripe the file-system when brining the Site 1 back online to ensure data is replicated.\<br><br>Simon<br><br>From: <<a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>>> on behalf of "<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>" <<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>><br>Reply-To: "<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>" <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Date: Thursday, 21 July 2016 at 13:45<br>To: "<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>" <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br><br>This is where my confusion sits. So if I have two sites, and two NDS Nodes per site with 1 NSD (to keep it simple), do I just present the physical LUN in Site1 to Site1 NDS Nodes and physical LUN in Site2 to Site2 NSD Nodes? Or is it that I present physical LUN in Site1 to all 4 NDS Nodes and the same at Site2? (Assuming SAN and not direct attached in this case). I know I?m being persistent but this for some reason confuses me.<br><br>Site1<br>NSD Node1<br> ---NSD1 ---Physical LUN1 from SAN1<br>NSD Node2<br><br><br>Site2<br>NSD Node3<br>---NSD2 ?Physical LUN2 from SAN2<br>NSD Node4<br><br><br>Or<br><br><br>Site1<br>NSD Node1<br> ----NSD1 ?Physical LUN1 from SAN1<br> ----NSD2 ?Physical LUN2 from SAN2<br>NSD Node2<br><br>Site 2<br>NSD Node3<br> ---NSD2 ? Physical LUN2 from SAN2<br> ---NSD1 --Physical LUN1 from SAN1<br>NSD Node4<br><br><br>Site 3<br>Node5 Quorum<br><br><br><br>From: <<a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>>> on behalf of Ken Hill <<a target="_blank" href="mailto:kenh@us.ibm.com">kenh@us.ibm.com</a><<a href="mailto:kenh@us.ibm.com">mailto:kenh@us.ibm.com</a>>><br>Reply-To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Date: Wednesday, July 20, 2016 at 7:02 PM<br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br><br>Yes - it is a cluster.<br><br>The sites should NOT be further than a MAN - or Campus network. If you're looking to do this over a large distance - it would be best to choose another GPFS solution (Multi-Cluster, AFM, etc).<br><br>Regards,<br><br>Ken Hill<br>Technical Sales Specialist | Software Defined Solution Sales<br>IBM Systems<br>________________________________<br><br>Phone:1-540-207-7270<br>E-mail: <a target="_blank" href="mailto:kenh@us.ibm.com">kenh@us.ibm.com</a><<a href="mailto:kenh@us.ibm.com">mailto:kenh@us.ibm.com</a>><br><br>[<a href="cid:image001.png@01D1E327.B037C650">cid:image001.png@01D1E327.B037C650</a>]<<a href="http://www.ibm.com/us-en/">http://www.ibm.com/us-en/</a>> [<a href="cid:image002.png@01D1E327.B037C650">cid:image002.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/lsf/">http://www-03.ibm.com/systems/platformcomputing/products/lsf/</a>> [<a href="cid:image003.png@01D1E327.B037C650">cid:image003.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/high-performance-services/index.html">http://www-03.ibm.com/systems/platformcomputing/products/high-performance-services/index.html</a>> [<a href="cid:image004.png@01D1E327.B037C650">cid:image004.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/symphony/index.html">http://www-03.ibm.com/systems/platformcomputing/products/symphony/index.html</a>> [<a href="cid:image005.png@01D1E327.B037C650">cid:image005.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/">http://www-03.ibm.com/systems/storage/spectrum/</a>> [<a href="cid:image006.png@01D1E327.B037C650">cid:image006.png@01D1E327.B037C650</a>] <<a href="http://www-01.ibm.com/software/tivoli/csi/cloud-storage/">http://www-01.ibm.com/software/tivoli/csi/cloud-storage/</a>> [<a href="cid:image007.png@01D1E327.B037C650">cid:image007.png@01D1E327.B037C650</a>] <<a href="http://www-01.ibm.com/software/tivoli/csi/backup-recovery/">http://www-01.ibm.com/software/tivoli/csi/backup-recovery/</a>> [<a href="cid:image008.png@01D1E327.B037C650">cid:image008.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/tape/ltfs/index.html">http://www-03.ibm.com/systems/storage/tape/ltfs/index.html</a>> [<a href="cid:image009.png@01D1E327.B037C650">cid:image009.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/">http://www-03.ibm.com/systems/storage/spectrum/</a>> [<a href="cid:image010.png@01D1E327.B037C650">cid:image010.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/scale/">http://www-03.ibm.com/systems/storage/spectrum/scale/</a>> <br> [<a href="cid:image011.png@01D1E327.B037C650">cid:image011.png@01D1E327.B037C650</a>] <<a href="https://www.ibm.com/marketplace/cloud/object-storage/us/en-us">https://www.ibm.com/marketplace/cloud/object-storage/us/en-us</a>><br><br>2300 Dulles Station Blvd<br>Herndon, VA 20171-6133<br>United States<br><br><br><br><br><br><br><br>From: "<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>" <<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>><br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Date: 07/20/2016 07:33 PM<br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br>Sent by: <a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>><br><br>________________________________<br><br><br><br>So in this scenario Ken, can server3 see any disks in site1?<br><br>From: <<a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>>> on behalf of Ken Hill <<a target="_blank" href="mailto:kenh@us.ibm.com">kenh@us.ibm.com</a><<a href="mailto:kenh@us.ibm.com">mailto:kenh@us.ibm.com</a>>><br>Reply-To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Date: Wednesday, July 20, 2016 at 4:15 PM<br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Subject: Re: [gpfsug-discuss] NDS in Two Site scenario<br><br><br> Site1 Site2<br> Server1 (quorum 1) Server3 (quorum 2)<br> Server2 Server4<br><br><br><br><br> SiteX<br> Server5 (quorum 3)<br><br><br><br><br>You need to set up another site (or server) that is at least power isolated (if not completely infrastructure isolated) from Site1 or Site2. You would then set up a quorum node at that site | location. This insures you can still access your data even if one of your sites go down.<br><br>You can further isolate failure by increasing quorum (odd numbers).<br><br>The way quorum works is: The majority of the quorum nodes need to be up to survive an outage.<br><br>- With 3 quorum nodes you can have 1 quorum node failures and continue filesystem operations.<br>- With 5 quorum nodes you can have 2 quorum node failures and continue filesystem operations.<br>- With 7 quorum nodes you can have 3 quorum node failures and continue filesystem operations.<br>- etc<br><br>Please see <a href="http://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/ibmspectrumscale42_content.html?view=kcfor">http://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/ibmspectrumscale42_content.html?view=kcfor</a> more information about quorum and tiebreaker disks.<br><br>Ken Hill<br>Technical Sales Specialist | Software Defined Solution Sales<br>IBM Systems<br>________________________________<br><br>Phone:1-540-207-7270<br>E-mail: <a target="_blank" href="mailto:kenh@us.ibm.com">kenh@us.ibm.com</a><<a href="mailto:kenh@us.ibm.com">mailto:kenh@us.ibm.com</a>><br><br>[<a href="cid:image012.png@01D1E327.B037C650">cid:image012.png@01D1E327.B037C650</a>]<<a href="http://www.ibm.com/us-en/">http://www.ibm.com/us-en/</a>> [<a href="cid:image013.png@01D1E327.B037C650">cid:image013.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/lsf/">http://www-03.ibm.com/systems/platformcomputing/products/lsf/</a>> [<a href="cid:image014.png@01D1E327.B037C650">cid:image014.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/high-performance-services/index.html">http://www-03.ibm.com/systems/platformcomputing/products/high-performance-services/index.html</a>> [<a href="cid:image015.png@01D1E327.B037C650">cid:image015.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/platformcomputing/products/symphony/index.html">http://www-03.ibm.com/systems/platformcomputing/products/symphony/index.html</a>> [<a href="cid:image016.png@01D1E327.B037C650">cid:image016.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/">http://www-03.ibm.com/systems/storage/spectrum/</a>> [<a href="cid:image017.png@01D1E327.B037C650">cid:image017.png@01D1E327.B037C650</a>] <<a href="http://www-01.ibm.com/software/tivoli/csi/cloud-storage/">http://www-01.ibm.com/software/tivoli/csi/cloud-storage/</a>> [<a href="cid:image018.png@01D1E327.B037C650">cid:image018.png@01D1E327.B037C650</a>] <<a href="http://www-01.ibm.com/software/tivoli/csi/backup-recovery/">http://www-01.ibm.com/software/tivoli/csi/backup-recovery/</a>> [<a href="cid:image019.png@01D1E327.B037C650">cid:image019.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/tape/ltfs/index.html">http://www-03.ibm.com/systems/storage/tape/ltfs/index.html</a>> [<a href="cid:image020.png@01D1E327.B037C650">cid:image020.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/">http://www-03.ibm.com/systems/storage/spectrum/</a>> [<a href="cid:image021.png@01D1E327.B037C650">cid:image021.png@01D1E327.B037C650</a>] <<a href="http://www-03.ibm.com/systems/storage/spectrum/scale/">http://www-03.ibm.com/systems/storage/spectrum/scale/</a>> <br> [<a href="cid:image022.png@01D1E327.B037C650">cid:image022.png@01D1E327.B037C650</a>] <<a href="https://www.ibm.com/marketplace/cloud/object-storage/us/en-us">https://www.ibm.com/marketplace/cloud/object-storage/us/en-us</a>><br><br>2300 Dulles Station Blvd<br>Herndon, VA 20171-6133<br>United States<br><br><br><br><br><br><br><br>From: "<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>" <<a target="_blank" href="mailto:Mark.Bush@siriuscom.com">Mark.Bush@siriuscom.com</a><<a href="mailto:Mark.Bush@siriuscom.com">mailto:Mark.Bush@siriuscom.com</a>>><br>To: gpfsug main discussion list <<a target="_blank" href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a><<a href="mailto:gpfsug-discuss@spectrumscale.org">mailto:gpfsug-discuss@spectrumscale.org</a>>><br>Date: 07/20/2016 04:47 PM<br>Subject: [gpfsug-discuss] NDS in Two Site scenario<br>Sent by: <a target="_blank" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>><br><br>________________________________<br><br><br><br><br>For some reason this concept is a round peg that doesn?t fit the square hole inside my brain. Can someone please explain the best practice to setting up two sites same cluster? I get that I would likely have two NDS nodes in site 1 and two NDS nodes in site two. What I don?t understand are the failure scenarios and what would happen if I lose one or worse a whole site goes down. Do I solve this by having scale replication set to 2 for all my files? I mean a single site I think I get it?s when there are two datacenters and I don?t want two clusters typically.<br><br><br><br>Mark R. Bush| Solutions Architect<br>Mobile: 210.237.8415 | <a target="_blank" href="mailto:mark.bush@siriuscom.com">mark.bush@siriuscom.com</a><<a href="mailto:mark.bush@siriuscom.com">mailto:mark.bush@siriuscom.com</a>><br>Sirius Computer Solutions | <a target="_blank" href="http://www.siriuscom.com">www.siriuscom.com</a><<a href="http://www.siriuscom.com/">http://www.siriuscom.com/</a>><br>10100 Reunion Place, Suite 500, San Antonio, TX 78216<br><br><br>This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. This message may be viewed by parties at Sirius Computer Solutions other than those named in the message header. This message does not contain an official representation of Sirius Computer Solutions. If you have received this communication in error, notify Sirius Computer Solutions immediately and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.<br><br>Sirius Computer Solutions<<a href="http://www.siriuscom.com/">http://www.siriuscom.com/</a>>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br><br> _______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br><br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment.html">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment.html</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image001.png<br>Type: image/png<br>Size: 1622 bytes<br>Desc: image001.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image002.png<br>Type: image/png<br>Size: 1598 bytes<br>Desc: image002.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0001.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0001.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image003.png<br>Type: image/png<br>Size: 1073 bytes<br>Desc: image003.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0002.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0002.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image004.png<br>Type: image/png<br>Size: 980 bytes<br>Desc: image004.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0003.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0003.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image005.png<br>Type: image/png<br>Size: 1565 bytes<br>Desc: image005.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0004.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0004.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image006.png<br>Type: image/png<br>Size: 1314 bytes<br>Desc: image006.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0005.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0005.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image007.png<br>Type: image/png<br>Size: 1169 bytes<br>Desc: image007.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0006.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0006.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image008.png<br>Type: image/png<br>Size: 1427 bytes<br>Desc: image008.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0007.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0007.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image009.png<br>Type: image/png<br>Size: 1370 bytes<br>Desc: image009.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0008.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0008.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image010.png<br>Type: image/png<br>Size: 1245 bytes<br>Desc: image010.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0009.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0009.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image011.png<br>Type: image/png<br>Size: 4455 bytes<br>Desc: image011.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0010.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0010.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image012.png<br>Type: image/png<br>Size: 1623 bytes<br>Desc: image012.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0011.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0011.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image013.png<br>Type: image/png<br>Size: 1599 bytes<br>Desc: image013.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0012.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0012.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image014.png<br>Type: image/png<br>Size: 1074 bytes<br>Desc: image014.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0013.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0013.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image015.png<br>Type: image/png<br>Size: 981 bytes<br>Desc: image015.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0014.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0014.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image016.png<br>Type: image/png<br>Size: 1566 bytes<br>Desc: image016.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0015.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0015.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image017.png<br>Type: image/png<br>Size: 1315 bytes<br>Desc: image017.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0016.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0016.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image018.png<br>Type: image/png<br>Size: 1170 bytes<br>Desc: image018.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0017.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0017.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image019.png<br>Type: image/png<br>Size: 1428 bytes<br>Desc: image019.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0018.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0018.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image020.png<br>Type: image/png<br>Size: 1371 bytes<br>Desc: image020.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0019.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0019.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image021.png<br>Type: image/png<br>Size: 1246 bytes<br>Desc: image021.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0020.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0020.png</a>><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: image022.png<br>Type: image/png<br>Size: 4456 bytes<br>Desc: image022.png<br>URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0021.png">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20160721/734c98ee/attachment-0021.png</a>><br><br>------------------------------<br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br><br><br>End of gpfsug-discuss Digest, Vol 54, Issue 48<br>**********************************************<br><br></font></div></div></div></font></font><BR>