[gpfsug-discuss] Confusing I/O Behavior

Uwe Falke UWEFALKE at de.ibm.com
Wed May 2 13:09:21 BST 2018


mmfsadm dump pgalloc
might get you one step further ...


 
Mit freundlichen Grüßen / Kind regards

 
Dr. Uwe Falke
 
IT Specialist
High Performance Computing Services / Integrated Technology Services / 
Data Center Services
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Rathausstr. 7
09111 Chemnitz
Phone: +49 371 6978 2165
Mobile: +49 175 575 2877
E-Mail: uwefalke at de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung: 
Thomas Wolter, Sven Schooß
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, 
HRB 17122 




From:   Peter Smith <peter.smith at framestore.com>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   02/05/2018 12:10
Subject:        Re: [gpfsug-discuss] Confusing I/O Behavior
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



"how do I see how much of the pagepool is in use and by what? I've looked 
at mmfsadm dump and mmdiag --memory and neither has provided me the 
information I'm looking for (or at least not in a format I understand)"

+1. Pointers appreciated! :-)

On 10 April 2018 at 17:22, Aaron Knister <aaron.s.knister at nasa.gov> wrote:
I wonder if this is an artifact of pagepool exhaustion which makes me ask 
the question-- how do I see how much of the pagepool is in use and by 
what? I've looked at mmfsadm dump and mmdiag --memory and neither has 
provided me the information I'm looking for (or at least not in a format I 
understand).

-Aaron

On 4/10/18 12:00 PM, Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP] 
wrote:
I hate admitting this but I?ve found something that?s got me stumped.

We have a user running an MPI job on the system. Each rank opens up 
several output files to which it writes ASCII debug information. The net 
result across several hundred ranks is an absolute smattering of teeny 
tiny I/o requests to te underlying disks which they don?t appreciate. 
Performance plummets. The I/o requests are 30 to 80 bytes in size. What I 
don?t understand is why these write requests aren?t getting batched up 
into larger write requests to the underlying disks.

If I do something like ?df if=/dev/zero of=foo bs=8k? on a node I see that 
the nasty unaligned 8k io requests are batched up into nice 1M I/o 
requests before they hit the NSD.

As best I can tell the application isn?t doing any fsync?s and isn?t doing 
direct io to these files.

Can anyone explain why seemingly very similar io workloads appear to 
result in well formed NSD I/O in one case and awful I/o in another?

Thanks!

-Stumped




_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-- 
Peter Smith · Senior Systems Engineer
London · New York · Los Angeles · Chicago · Montréal 
T  +44 (0)20 7208 2600 · M  +44 (0)7816 123009 
28 Chancery Lane, London WC2A 1LB 
Twitter · Facebook · framestore.com

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss







More information about the gpfsug-discuss mailing list