[gpfsug-discuss] Spectrum Scale Ganesha NFS multi threaded AFM?

[Tomer Perry]

Hi,

I believe the right term is not multithreaded, but rather multistream. NFS 
will submit multiple requests in parallel, but without using large enough 
window you won't be able to get much of each stream.
So, the first place to look is here: 
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1adm_tuningbothnfsclientnfsserver.htm 
 - and while its talking about "Kernel NFS" the same apply to any TCP 
socket based communication ( including Ganesha). I tend to test the 
performance using iperf/nsdperf ( just make sure to use single stream) in 
order to see what is the expected maximum performance.
After that, you can start looking into "how can I get multiple streams?" - 
for that there are two options:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1ins_paralleldatatransfersafm.htm 

and
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/b1lins_afmparalleldatatransferwithremotemounts.htm 


The former enhance large file transfer, while the latter ( new in 5.0.4) 
will help with multiple small files as well.



Regards,

Tomer Perry
Scalable I/O Development (Spectrum Scale)
email: tomp at il.ibm.com
1 Azrieli Center, Tel Aviv 67021, Israel
Global Tel:    +1 720 3422758
Israel Tel:      +972 3 9188625
Mobile:         +972 52 2554625




From:   Andi Christiansen <andi at christiansen.xxx>
To:     "gpfsug-discuss at spectrumscale.org" 
<gpfsug-discuss at spectrumscale.org>
Date:   21/02/2020 15:25
Subject:        [EXTERNAL] [gpfsug-discuss] Spectrum Scale Ganesha NFS 
multi threaded AFM?
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



Hi all, 

i have searched the internet for a good time now with no answer to this.. 
So i hope someone can tell me if this is possible or not. 

We use NFS from our Cluster1 to a AFM enabled fileset on Cluster2. That is 
working as intended. But when AFM transfers files from one site to another 
it caps out at about 5-700Mbit/s which isnt impressive.. The sites are 
connected on 10Gbit links but the distance/round-trip is too far/high to 
use the NSD protocol with AFM. 

On the cluster where the fileset is exported we can only see 1 session 
against the client cluster, is there a way to either tune Ganesha or AFM 
to use more threads/sessions? 

We have about 7.7Gbit bandwidth between the sites from the 10Gbit links 
and with multiple NFS sessions we can reach the maximum bandwidth(each 
using about 50-60MBits per session). 

Best Regards 
Andi Christiansen _______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=mLPyKeOa1gNDrORvEXBgMw&m=XKMIdSqQ76jf_FrIRFtAhMsgU-MkPFhxBJjte8AdeYs&s=vih7W_XcatoqN_MhS3gEK9RR6RxpNrfB2UvvQeXqyH8&e= 





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200221/74c5afeb/attachment-0002.htm>