<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Nur Text Zchn";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.NurTextZchn
{mso-style-name:"Nur Text Zchn";
mso-style-priority:99;
mso-style-link:"Nur Text";
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="DE">Hello!<o:p></o:p></span></p>
<p class="MsoNormal">We are designing two fs where we cannot anticipate if there will be 3000, or maybe 5000 or more nodes totally accessing these filesystems<o:p></o:p></p>
<p class="MsoNormal">What we saw, was that execution time of mmdf can last 5-7min <o:p></o:p></p>
<p class="MsoNormal">We openend a case and they said, that during such commands like mmdf or also mmfsck, mmdefragfs,mmresripefs all regions must be scanned at this is the reason why it takes so long<o:p></o:p></p>
<p class="MsoNormal">The technichian also said, that it is “rule of thumb” that there should be
<o:p></o:p></p>
<p class="MsoNormal">(-n)*32 regions , this would then be enough ( N=5000 <span style="font-family:Wingdings">
à</span> 160000 regions per pool ?) <o:p></o:p></p>
<p class="MsoNormal">(also Block size has influence on regions ?) <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">#mmfsadm saferdump stripe <o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Gives the regions number
<o:p></o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""> storage pools: max 8<o:p></o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""> alloc map type 'scatter'<o:p></o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""> 0: name 'system' Valid nDisks 12 nInUse 12 id 0 poolFlags 0 thinProvision reserved inode -1, reserved nBlocks 0<o:p></o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span style="font-family:"Courier New""> <u>regns 170413</u> segs 1 size 4096 FBlks 0 MBlks 3145728 subblock size 8192<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We also saw when creating the filesystem with a speciicic (-n) very high (5000) (where mmdf execution time was some minutes) and then changing (-n) to a lower value this does not influence the behavior any more<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My question is: Is the rule (Number of Nodes)x5000 for number of regios in a pool an good estimation , <o:p></o:p></p>
<p class="MsoNormal">Is it better to overestimate the number of Nodes (lnger running commands) or is it unrealistic to get into problems when not reaching the regions number calculated ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Does anybody have experience with high number of nodes (>>3000) and how to design the filesystems for such large clusters ?
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thank you very much in advance !<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="DE-AT" style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b><br>
</span><b><i><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray">Technical Consultant</span></i></b><span lang="DE-AT" style="color:black">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE-AT" style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: <a href="mailto:sklenka@edv-design.at" target="_blank"><span style="color:#0563C1">sklenka@edv-design.at</span></a></span><span lang="DE-AT" style="font-size:8.0pt;color:black"><br>
</span><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">Internet:
<a href="http://www.edv-design.at/" target="_blank"><span style="color:#0563C1">www.edv-design.at</span></a></span><i><span lang="DE-AT" style="font-size:8.0pt;color:black"><br>
<br>
</span></i><span lang="DE"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE"><o:p> </o:p></span></p>
</div>
</body>
</html>