<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Just a few thoughts:</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We've often increased the seqDiscardThreshold in SAS on AIX GPFS implementations to much larger values ( a lot of large file sequential I/O). With a pagepool of 64G, maybe set to 4 or 8GB?</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Sequential writes from the application when spread across mutliple LUNs on the storage+Scale often do not resemble sequential writes from the storage POV.</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
If overrunning a disk device representation in the OS, the device queue depth is one parameter that might be expanded to offer some relief.</div>
<div id="Signature">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Arial, Helvetica, sans-serif;">---</span></div>
<div><span style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Glen Corneau</span></div>
<div><span style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Senior, Power Partner Technical Specialist (PTS-P)</span></div>
<div><span style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">IBM Technology, North America</span></div>
<div><span style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Email: gcorneau@us.ibm.com</span></div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Cell: 512-420-7988</span></div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> on behalf of Michal Hruška <Michal.Hruska@mcomputers.cz><br>
<b>Sent:</b> Thursday, February 8, 2024 08:59<br>
<b>To:</b> gpfsug-discuss@gpfsug.org <gpfsug-discuss@gpfsug.org><br>
<b>Subject:</b> [EXTERNAL] Re: [gpfsug-discuss] sequential I/O write - performance</font>
<div> </div>
</div>
<div>
<div style="display:none!important; display:none; visibility:hidden; font-size:1px; color:#ffffff; line-height:1px; height:0px; max-height:0px; opacity:0; overflow:hidden">
@Aaron Yes, I can confirm that 2MB blocks are transfered over. @ Jan-Frode We tried to change multiple parameters, but if you know the best combination for sequential IO, please let me know. #mmlsconfig autoload no dmapiFileHandleSize 32 minReleaseLevel
</div>
<div style="display:none!important; display:none; visibility:hidden; font-size:1px; color:#ffffff; line-height:1px; height:0px; max-height:0px; opacity:0; overflow:hidden">
</div>
<style>
<!--
#x_pfptBannereh33fvq
{display:block!important;
visibility:visible!important;
opacity:1!important;
background-color:#D0D8DC!important;
max-width:none!important;
max-height:none!important}
-->
</style>
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Verdana}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
span.x_StylE-mailovZprvy17
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{}
@page WordSection1
{margin:70.85pt 70.85pt 70.85pt 70.85pt}
div.x_WordSection1
{}
-->
</style>
<div class="x_WordSection1">
<p class="x_MsoNormal">@Aaron</p>
<p class="x_MsoNormal">Yes, I can confirm that 2MB blocks are transfered over.<br>
<br>
</p>
<p class="x_MsoNormal">@ Jan-Frode</p>
<p class="x_MsoNormal">We tried to change multiple parameters, but if you know the best combination for sequential IO, please let me know.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">#mmlsconfig</p>
<p class="x_MsoNormal">autoload no</p>
<p class="x_MsoNormal">dmapiFileHandleSize 32</p>
<p class="x_MsoNormal">minReleaseLevel 5.1.9.0</p>
<p class="x_MsoNormal">tscCmdAllowRemoteConnections no</p>
<p class="x_MsoNormal">ccrEnabled yes</p>
<p class="x_MsoNormal">cipherList AUTHONLY</p>
<p class="x_MsoNormal">sdrNotifyAuthEnabled yes</p>
<p class="x_MsoNormal">pagepool 64G</p>
<p class="x_MsoNormal">maxblocksize 16384K</p>
<p class="x_MsoNormal">maxMBpS 40000</p>
<p class="x_MsoNormal">maxReceiverThreads 32</p>
<p class="x_MsoNormal">nsdMaxWorkerThreads 512</p>
<p class="x_MsoNormal">nsdMinWorkerThreads 8</p>
<p class="x_MsoNormal">nsdMultiQueue 256</p>
<p class="x_MsoNormal">nsdSmallThreadRatio 0</p>
<p class="x_MsoNormal">nsdThreadsPerQueue 3</p>
<p class="x_MsoNormal">prefetchAggressiveness 2</p>
<p class="x_MsoNormal">adminMode central</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">/dev/fs0<br>
<br>
</p>
<p class="x_MsoNormal">@Uwe</p>
<p class="x_MsoNormal">Using iohist we found out that gpfs is overloading one dm-device (it took about 500ms to finish IOs). We replaced the „problematic“ dm-device (as we have enough drives to play with) for new one but the overloading issue just jumped to
another dm-device.<br>
We believe that this behaviour is caused by the gpfs but we are unable to locate the root cause of it.</p>
<p class="x_MsoNormal"> </p>
<div>
<div>
<p class="x_MsoNormal"><span style="font-size:9.0pt; font-family:"Verdana",sans-serif; color:#595959">Best,<br>
Michal</span></p>
</div>
</div>
<p class="x_MsoNormal"> </p>
</div>
</div>
</body>
</html>