<div dir="ltr"><div>a few more weeks and we have a better answer than dump pgalloc ;-)</div><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, May 2, 2018 at 6:07 AM Peter Smith <<a href="mailto:peter.smith@framestore.com">peter.smith@framestore.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">"how do <span class="m_-3989759872882691617gmail-il">I</span> see how much of the pagepool is in use and by what? <span class="m_-3989759872882691617gmail-il">I</span>'ve looked at mmfsadm dump and mmdiag --memory and neither has provided me the information <span class="m_-3989759872882691617gmail-il">I</span>'m looking for (or at least not in a format <span class="m_-3989759872882691617gmail-il">I</span> understand)"<br><br></div></div><div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">+1. Pointers appreciated! :-)<br></div></div><div class="gmail_extra"></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 April 2018 at 17:22, Aaron Knister <span dir="ltr"><<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I wonder if this is an artifact of pagepool exhaustion which makes me ask the question-- how do I see how much of the pagepool is in use and by what? I've looked at mmfsadm dump and mmdiag --memory and neither has provided me the information I'm looking for (or at least not in a format I understand).<br>
<br>
-Aaron<br>
<br>
On 4/10/18 12:00 PM, Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP] wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I hate admitting this but I’ve found something that’s got me stumped.<br>
<br>
We have a user running an MPI job on the system. Each rank opens up several output files to which it writes ASCII debug information. The net result across several hundred ranks is an absolute smattering of teeny tiny I/o requests to te underlying disks which they don’t appreciate. Performance plummets. The I/o requests are 30 to 80 bytes in size. What I don’t understand is why these write requests aren’t getting batched up into larger write requests to the underlying disks.<br>
<br>
If I do something like “df if=/dev/zero of=foo bs=8k” on a node I see that the nasty unaligned 8k io requests are batched up into nice 1M I/o requests before they hit the NSD.<br>
<br>
As best I can tell the application isn’t doing any fsync’s and isn’t doing direct io to these files.<br>
<br>
Can anyone explain why seemingly very similar io workloads appear to result in well formed NSD I/O in one case and awful I/o in another?<br>
<br>
Thanks!<br>
<br>
-Stumped<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br><span class="m_-3989759872882691617HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_-3989759872882691617HOEnZb"><font color="#888888">
<br>
-- <br>
Aaron Knister<br>
NASA Center for Climate Simulation (Code 606.2)<br>
Goddard Space Flight Center<br>
<a href="tel:(301)%20286-2776" value="+13012862776" target="_blank">(301) 286-2776</a><br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</font></span></blockquote></div><br><br clear="all"><br></div><div class="gmail_extra">-- <br><div class="m_-3989759872882691617gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">
<img style="display:block" src="http://d2ebqvzxkz5e0j.cloudfront.net/img/fs-email-signature-logo.png" alt="Framestore" height="34" width="143"> <span style="color:rgb(61,133,198)"><span style="font-weight:bold">Peter Smith</span> · Senior Systems Engineer<br><span style="font-weight:bold">London</span> · <span>New York</span> · <span>Los Angeles</span> · <span>Chicago</span> · <span>Montréal</span></span>
<br>
T <a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap">+44 (0)20 7208 2600</a> · M <a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap" href="tel:+44%20%280%297816%20123009" target="_blank">+44 (0)7816 123009</a>
<br>
<a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap;font-style:italic" href="https://www.google.co.uk/maps/place/19-23+Wells+Street,+London+W1T+3PQ" target="_blank">28 Chancery Lane, London WC2A 1LB</a>
<br>
<a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap;border-bottom:1px solid rgb(0,117,201)" href="https://twitter.com/framestore" target="_blank">Twitter</a> · <a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap;border-bottom:1px solid rgb(0,117,201)" href="https://www.facebook.com/framestore" target="_blank">Facebook</a> · <a style="color:rgb(0,117,201)!important;text-decoration:none!important;white-space:nowrap;border-bottom:1px solid rgb(0,117,201)" href="http://www.framestore.com" target="_blank">framestore.com</a><br><br><img src="https://s3-eu-west-1.amazonaws.com/front-media/assets/c0/e8/3a/e4/emailfooter2018.gif"><br></div></div></div></div></div></div></div></div></div></div>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div></div></div>