<div dir="ltr">have you tried to just leave lrocInodes and lrocDirectories on and turn data off ?<div>also did you increase maxstatcache so LROC actually has some compact objects to use ?</div><div>if you send value for maxfilestocache,maxfilestocache,workerthreads and available memory of the node i can provide a start point.</div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jan 25, 2017 at 10:20 PM Matt Weil <<a href="mailto:mweil@wustl.edu">mweil@wustl.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg">
<p class="gmail_msg"><br class="gmail_msg">
</p>
<br class="gmail_msg">
<div class="m_-8342588277744312912moz-cite-prefix gmail_msg">On 1/25/17 3:00 PM, Sven Oehme wrote:<br class="gmail_msg">
</div>
<blockquote type="cite" class="gmail_msg">
<div dir="ltr" class="gmail_msg">Matt,
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">the assumption was that the remote devices are slower than LROC. there is some attempts in the code to not schedule more than a maximum numbers of outstanding i/os to the LROC device, but this doesn't help in all cases and is depending on what kernel level
parameters for the device are set. the best way is to reduce the max size of data to be cached into lroc.
<br class="gmail_msg">
</div>
</div>
</blockquote></div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg">
I just turned LROC file caching completely off. most if not all of the IO is metadata. Which is what I wanted to keep fast. It is amazing once you drop the latency the IO's go up way more than they ever where before. I guess we will need another nvme.</div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg"><br class="gmail_msg">
<blockquote type="cite" class="gmail_msg">
<div dir="ltr" class="gmail_msg">
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">sven</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
</div>
<br class="gmail_msg">
<div class="gmail_quote gmail_msg">
<div dir="ltr" class="gmail_msg">On Wed, Jan 25, 2017 at 9:50 PM Matt Weil <<a href="mailto:mweil@wustl.edu" class="gmail_msg" target="_blank">mweil@wustl.edu</a>> wrote:<br class="gmail_msg">
</div>
<blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello all,<br class="gmail_msg">
<br class="gmail_msg">
We are having an issue where the LROC on a CES node gets overrun 100%<br class="gmail_msg">
utilized. Processes then start to backup waiting for the LROC to<br class="gmail_msg">
return data. Any way to have the GPFS client go direct if LROC gets to<br class="gmail_msg">
busy?<br class="gmail_msg">
<br class="gmail_msg">
Thanks<br class="gmail_msg">
Matt<br class="gmail_msg">
<br class="gmail_msg">
________________________________<br class="gmail_msg">
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action
in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.<br class="gmail_msg">
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">
spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote>
</div>
<br class="gmail_msg">
<fieldset class="m_-8342588277744312912mimeAttachmentHeader gmail_msg"></fieldset> <br class="gmail_msg">
<pre class="gmail_msg">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" class="gmail_msg" target="_blank">spectrumscale.org</a>
<a class="m_-8342588277744312912moz-txt-link-freetext gmail_msg" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br class="gmail_msg">
<span style="font-size:10.0pt;font-family:"Arial","sans-serif"" class="gmail_msg"><u class="gmail_msg"></u><u class="gmail_msg"></u></span>
<p class="gmail_msg"></p>
<div class="gmail_msg"></div>
<p class="MsoNormal gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
<div class="MsoNormal gmail_msg" align="center" style="text-align:center">
<hr size="2" width="100%" align="center" class="gmail_msg">
</div>
<p class="MsoNormal gmail_msg"><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:gray" class="gmail_msg">The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended
recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone
or return mail.</span></p>
</div>
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote></div>