[gpfsug-discuss] afmRefreshAsync questions

Venkateswara R Puvvada vpuvvada at in.ibm.com
Sat Oct 5 05:30:49 BST 2019


I would recommend opening a case, collect the default traces from both 
gateway and application (or protocol) nodes to check the RPC overhead. 
There should not be difference between mmap IO and regular IO for AFM 
filesets. Also note that refresh intervals are stored as part of inode and 
for the large number of file access it is possible that inodes are evicted 
as part of dcache shrinkage and next access to the same files might go to 
home for the revalidation. afmRefreshAsync  option can be set at fleset 
level also. Looks like it is missing from the documentation, this will be 
corrected.

~Venkat (vpuvvada at in.ibm.com)



From:   Andreas Mattsson <andreas.mattsson at maxiv.lu.se>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   10/03/2019 07:25 PM
Subject:        [EXTERNAL] Re: [gpfsug-discuss] afmRefreshAsync questions
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



After further investigaion, it seems like this XDS software is using 
memory mapped io when operating on the files.
Is it possible that MMAP IO has a higher performance hit by AFM than 
regular file access?

/Andreas
____________________________________________


Andreas Mattsson
Systems Engineer
 
MAX IV Laboratory
Lund University
P.O. Box 118, SE-221 00 Lund, Sweden
Visiting address: Fotongatan 2, 224 84 Lund
Mobile: +46 706 64 95 44
andreas.mattsson at maxiv.lu.se
www.maxiv.se


Från: gpfsug-discuss-bounces at spectrumscale.org 
<gpfsug-discuss-bounces at spectrumscale.org> för Andreas Mattsson 
<andreas.mattsson at maxiv.lu.se>
Skickat: den 1 oktober 2019 08:33:35
Till: gpfsug main discussion list
Ämne: Re: [gpfsug-discuss] afmRefreshAsync questions 
 
Hi,
I've tried increasing all the refresh intervals, but even at 300 seconds, 
there is very little performance increase.
The job runs in several steps, and gets held up at two places, as far as I 
can see.
First at a kind of parallelisation step where about 1000-3000 files are 
created in the current working folder on a single compute node, and then 
at a step where lots of small output files are written on each of the 
compute nodes involved in the job. 
Comparing with running the same data set on a non-AFM cache fileset in the 
same storage system, it runs at least a factor 5 slower, even with really 
high refresh intervals.
In the Scale documentation, it states that the afmRefreshAsync is only 
configurable cluster wide. Is it also configurable on a per-fileset level?
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.3/com.ibm.spectrum.scale.v5r03.doc/bl1adm_configurationparametersAFM.htm


The software is XDS,  http://xds.mpimf-heidelberg.mpg.de/
Unfortunately it is a closed source software, so it is not possible to 
adapt the software.

Regards,
Andreas Mattsson

____________________________________________


Andreas Mattsson
Systems Engineer
 
MAX IV Laboratory
Lund University
P.O. Box 118, SE-221 00 Lund, Sweden
Visiting address: Fotongatan 2, 224 84 Lund
Mobile: +46 706 64 95 44
andreas.mattsson at maxiv.lu.se
www.maxiv.se


Från: gpfsug-discuss-bounces at spectrumscale.org 
<gpfsug-discuss-bounces at spectrumscale.org> för Venkateswara R Puvvada 
<vpuvvada at in.ibm.com>
Skickat: den 27 september 2019 10:23:13
Till: gpfsug main discussion list
Ämne: Re: [gpfsug-discuss] afmRefreshAsync questions 
 
Hi,

Both storage and client clusters  have to be on 5.0.3.x to get the AFM 
revalidation performance with afmRefreshAsync. What are the refresh 
intervals ?, you could also try increasing them. Is this config option set 
at fileset level or cluster level ?

~Venkat (vpuvvada at in.ibm.com)



From:        Andreas Mattsson <andreas.mattsson at maxiv.lu.se>
To:        GPFS User Group <gpfsug-discuss at spectrumscale.org>
Date:        09/26/2019 03:26 PM
Subject:        [EXTERNAL] [gpfsug-discuss] afmRefreshAsync questions
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



Hi,
Due to having a data analysis software that isn't running well at all in 
our AFM caches, it runs 4-6 times slower on an AFM cache than on a non-AFM 
fileset on the same storage system, I wanted to try out the 
afmRefreshAsync feature that came with 5.0.3 to see if it is the cache 
data refresh that is holding things up.
Enabling this feature has had zero impact on performance of the software 
though.

The storage cluster is running 5.0.3.x, and afmRefreshAsync has been set 
there, but at the moment the remote-mounting client cluster is still 
running 5.0.2.x.
Would this feature still have any effect in this setup?

Regards,
Andreas Mattsson

____________________________________________


Andreas Mattsson
Systems Engineer
 
MAX IV Laboratory
Lund University
P.O. Box 118, SE-221 00 Lund, Sweden
Visiting address: Fotongatan 2, 224 84 Lund
Mobile: +46 706 64 95 44
andreas.mattsson at maxiv.lu.se
www.maxiv.se
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=92LOlNh2yLzrrGTDA7HnfF8LFr55zGxghLZtvZcZD7A&m=vrw7qt4uEH-dBuEZSxUvPQM-SJOC0diQptL6vnfxCQA&s=rbRvqgv05seDPo5wFgK2jlRkzvHtU7y7zoNQ3rDV0d0&e= 





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191005/04202adc/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4232 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191005/04202adc/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4232 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191005/04202adc/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4232 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191005/04202adc/attachment-0008.png>


More information about the gpfsug-discuss mailing list