<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Folks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
So, we are experiencing some very long IO waiters in our GPFS cluster:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">#<span class="Apple-converted-space">
</span>mmdiag --waiters<span class="Apple-converted-space"> </span></span></p>
<p class="p2" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139);min-height:20px">
<span class="s1" style="font-variant-ligatures:no-common-ligatures"></span><br>
</p>
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">=== mmdiag: waiters ===</span></p>
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">Waiting 17.3823 sec since 10:41:01, monitored, thread 21761 NSDThread: for I/O completion</span></p>
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">Waiting 16.6140 sec since 10:41:02, monitored, thread 21730 NSDThread: for I/O completion</span></p>
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">Waiting 15.3004 sec since 10:41:03, monitored, thread 21763 NSDThread: for I/O completion</span></p>
<p class="p1" style="margin:0px;font:17px Menlo;background-color:rgb(254, 244, 139)">
<span class="s1" style="font-variant-ligatures:no-common-ligatures">Waiting 15.2013 sec since 10:41:03, monitored, thread 22175 <br>
</span></p>
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
However, GPFS support is pointing to our <span style="background-color:rgb(255, 255, 255);display:inline !important">IBM Storwize V5030</span> disk system as the source of latency. Unfortunately, we don't have paid support for the system so we are polling for
anyone who might be able to assist.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Does anyone by chance have any experience with IBM Storwize V5030 or possess a<span style="background-color:rgb(255, 255, 255);display:inline !important"> problem determination guide for the V5030?</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We've briefly reviewed the V5030 management portal, but we still haven't identified a cause for the increased latencies (i.e. read ~129ms, write ~198ms).
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Granted, we have some heavy client workloads, yet we seem to experience this drastic drop in performance every couple of months, probably exacerbated by heavy IO demands.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Any assistance would be much appreciated.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div></div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p style="margin-top:0px; margin-bottom:0px"></p>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<span style="font-family:Arial,sans-serif; font-size:13.3333px">Thanks,</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<font face="Calibri,Arial,Helvetica,sans-serif"><br>
</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<font face="Calibri,Arial,Helvetica,sans-serif">
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<b><span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black">Oluwasijibomi (Siji) Saula</span></b></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black">HPC Systems Administrator / Information Technology</span><span style="margin:0px; font-family:"Times New Roman",serif; color:black"></span></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black"> </span><span style="margin:0px; font-family:"Times New Roman",serif; color:black"></span></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black"><span style="font-family:Arial,sans-serif; background-color:rgb(255,255,255); display:inline!important">Research 2 Building</span> 220B / Fargo ND 58108-6050</span></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black">p: 701.231.7749 /<span> </span><a href="http://www.ndsu.edu/" target="_blank" rel="noopener noreferrer" style="margin:0px"><span style="margin:0px; color:purple">www.ndsu.edu</span></a></span><span style="margin:0px; font-family:"Times New Roman",serif; color:black"></span></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; font-size:10pt; font-family:Arial,sans-serif; color:black"> </span><span style="margin:0px; font-family:"Times New Roman",serif; color:black"></span></p>
<p style="color:rgb(32,31,30); text-align:start; background-color:rgb(255,255,255); font-size:11pt; font-family:Calibri,sans-serif; margin:0px">
<span style="margin:0px; color:rgb(31,73,125)"><img class="EmojiInsert" style="margin:0px; width:140.99pt; height:26.24pt" data-outlook-trace="F:0|T:1" src="cid:image001.gif@01D57DE0.91C300C0"></span></p>
</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<br>
<p style="margin-top:0px; margin-bottom:0px"></p>
</div>
</div>
</div>
</div>
</body>
</html>