<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10pt" ><div dir="ltr" >Hi Lohit,</div>
<div dir="ltr" > </div>
<div dir="ltr" >First, a few questions to understand better your problem:</div>
<div dir="ltr" >- What is the minimum release level of both clusters?</div>
<div dir="ltr" >- What is the version of filesystem layout for 16MB, 1MB and 512KB?</div>
<div dir="ltr" >- What is the subblocksize of each filesystem?</div>
<div dir="ltr" >- How many IOPS, block size and throughput are you doing on each filesystem?</div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" ><div> </div>
<table cellpadding="0" cellspacing="0" style="border:0;table-layout:auto;white-space:normal;color:#5f5f5f;width:650px;font-size:7.2pt" width="650px" > <tbody> <tr> <td style="vertical-align: bottom;" > <div style="font-size:9pt" ><span style="font-size:1.000em;" >Abraços / Regards / Saludos,</span></div>
<p> </p>
<div class="vcard" style="margin:0 0 8px 0" ><span style="font-size:1.000em;" ><b class="fn n" style="color:#888888;font-size:12pt" ><span class="given-name" >Anderson</span> <span class="family-name" >Nobre</span></b><br> Power and Storage Consultant<br> IBM Systems Hardware Client Technical Team – IBM Systems Lab Services<br> <br> <img alt="community_general_lab_services" class="org" height="120" src="http://esign.webmaster.ibm.com/img/signature_lab_services.jpg" width="333" ></span></div> </td> <td> </td> </tr> </tbody></table>
<table cellpadding="0" cellspacing="0" style="border:0;table-layout:auto;white-space:normal;color:#5f5f5f;width:650px;font-size:7.2pt" width="650px" > <tbody> <tr> <td colspan="3" > <hr style="border:0;color:#696969;background-color:#696969;height:1px;" ></td> </tr> <tr> <td style="vertical-align: top;font-family:arial,helvetica,sans-serif" width="400px" ><span style="font-size:1.000em;" ><span class="tel" ><span style="color:#466BB0;font-weight:bold" ><abbr class="type" title="work" >Phone:</abbr></span> <span class="value" >55-19-2132-4317</span></span><br> <span style="color:#466BB0;font-weight:bold;font-style:7.2pt" >E-mail:</span> <a class="email" href="mailto:anobre@br.ibm.com" target="_blank"><span class="value" style="color:#5f5f5f" >anobre@br.ibm.com</span></a></span></td> <td align="right" style="vertical-align: top;font-family:arial,helvetica,sans-serif" width="150px" ><span style="font-size:1.000em;" ><img alt="IBM" class="org" src="http://esign.webmaster.ibm.com/img/ibm3.gif" ></span></td> </tr> </tbody></table></div></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Valleru, Lohit/Information Systems" <valleru@cbio.mskcc.org><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug-discuss@spectrumscale.org<br>Cc:<br>Subject: [EXTERNAL] [gpfsug-discuss] Maxblocksize tuning alternatives/max number of buffers<br>Date: Fri, Feb 28, 2020 12:30<br>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >Hello Everyone,<br><br>I am looking for alternative tuning parameters that could do the same job as tuning the maxblocksize parameter.<br><br>One of our users run a deep learning application on GPUs, that does the following IO pattern:<br><br>It needs to read random small sections about 4K in size from about 20,000 to 100,000 files of each 100M to 200M size.<br><br>When performance tuning for the above application on a 16M filesystem and comparing it to various other file system block sizes - I realized that the performance degradation that I see might be related to the number of buffers.<br><br>I observed that the performance varies widely depending on what maxblocksize parameter I use.<br>For example, using a 16M maxblocksize for a 512K or a 1M block size filesystem differs widely from using a 512K or 1M maxblocksize for a 512K or a 1M block size filesystem.<br><br>The reason I believe might be related to the number of buffers that I could keep on the client side, but I am not sure if that is the all that the maxblocksize is affecting.<br><br>We have different file system block sizes in our environment ranging from 512K, 1M and 16M.<br><br>We also use storage clusters and compute clusters design.<br><br>Now in order to mount the 16M filesystem along with the other filesystems on compute clusters - we had to keep the maxblocksize to be 16M - no matter what the file system block size.<br><br>I see that I get maximum performance for this application from a 512K block size filesystem and a 512K maxblocksize.<br>However, I will not be able to mount this filesystem along with the other filesystems because I will need to change the maxblocksize to 16M in order to mount the other filesystems of 16M block size.<br><br>I am thinking if there is anything else that can do the same job as maxblocksize parameter.<br><br>I was thinking about the parameters like maxBufferDescs for a 16M maxblocksize, but I believe it would need a lot more pagepool to keep the same number of buffers as would be needed for a 512k maxblocksize.<br><br>May I know if there is any other parameter that could help me the same as maxblocksize, and the side effects of the same?<br><br>Thank you,<br>Lohit<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </font><br> </div></blockquote>
<div dir="ltr" > </div></div><BR>