<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:362635915;
mso-list-template-ids:-1712563738;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">There is no advantage to having more failure group than maximum number of replica supported by a file system plus 1 for tie breaker disks. In a multiple site setup, you will want 1 failure group per site in order to ensure 1 replica
is placed at each site as GPFS will place replica using round-robin amount the failure groups.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> on behalf of Luke Sudbery <l.r.sudbery@bham.ac.uk><br>
<b>Date: </b>Tuesday, March 18, 2025 at 9:28</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;color:black">AM<br>
<b>To: </b>gpfsug-discuss@gpfsug.org <gpfsug-discuss@gpfsug.org><br>
<b>Subject: </b>[EXTERNAL] [gpfsug-discuss] Replicated cluster - failure groups<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">We are planning a replicated cluster. Due to a combination of purchasing cycles, floor loading and VAT-exemption status for half the equipment/data, this will be
built over time using a total 8 Lenovo DSS building blocks. 2 main pools, in 2<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">We are planning a replicated cluster. Due to a combination of purchasing cycles, floor loading and VAT-exemption status for half the equipment/data, this will be built over time using a total 8 Lenovo DSS building blocks. 2 main pools,
in 2 data centres, with 2 DSSG per pool, and a quorum/manager node with a local tie breaker disk in a 3rd physical location.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">My main question is about failure groups - so far, with 2 DSS and 1 tiebreaker, we would have had 1 failure group per DSS and 1 for the tie breaker disk, giving us a total of 3. But if we did that now we would have 9 failure groups in 1
filesystem, which is more than the maximum number of replicas of the file system descriptor and not desirable, as I understand it.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">So we could have either:<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">1 FG per physical site, and assign all 4 DSS per site to 1 FG, and a 3rd to the tiebreaker<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">1 FG per pool per site, with 2 DSS in each FG. This makes sense as the pairs of DSSG will both always need to be up for all the data in the pool to be accessible.
<o:p></o:p></li></ul>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">The second option would give us 5 failure groups, but what would be the advantage and disadvantages of more failure groups?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Many thanks,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Luke<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">-- </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">Luke Sudbery</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">Principal Engineer (HPC and Storage).</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">Architecture, Infrastructure and Systems</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">Advanced Research Computing, IT Services</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D">Room 132, Computer Centre G5, Elms Road</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;color:#1F497D">Please note I don’t work on Monday.</span></b><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>