<font size=2 face="sans-serif">Hello Jake,</font><br><br><font size=2 face="sans-serif">You will have to set the mapping to
include all the GW's that you want to involve in the transfer. Please refer
to the example provided in the Knowledge Centre:</font><br><a href=http://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.1/com.ibm.spectrum.scale.v4r21.doc/bl1ins_paralleldatatransfersafm.htm><tt><font size=2 color=blue>http://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.1/com.ibm.spectrum.scale.v4r21.doc/bl1ins_paralleldatatransfersafm.htm</font></tt></a><br><font size=2 face="sans-serif"><br><br><br>Thanks and Regards<br>Radhika<br><br></font><tt><font size=2>Message: 1<br>Date: Sun, 13 Nov 2016 14:18:38 +0000<br>From: Jake Carroll <jake.carroll@uq.edu.au><br>To: "gpfsug-discuss@spectrumscale.org"<br>
<gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] Achieving high parallelism with AFM using<br>
NFS?<br>Message-ID: <025F8914-F7A0-465F-9B99-961F70DA2B03@uq.edu.au><br>Content-Type: text/plain; charset="utf-8"<br><br>Hi all.<br><br>After some help from IBM, we?ve concluded (and been told) that AFM over
the NSD protocol when latency is greater than around 50ms on the RTT is
effectively unusable. We?ve proven that now, so it is time to move on from
the NSD protocol being an effective option in those conditions (unless
IBM can consider it something worthy of an RFE and can fix it!).<br><br>The problem we face now, is one of parallelism and filling that 10GbE/40GbE/100GbE
pipe efficiently, when using NFS as the transport provider for AFM.<br><br>On my test cluster at ?Cache? side I?ve got two or three gateways:<br><br>[root@mc-5 ~]# mmlscluster<br><br>GPFS cluster information<br>========================<br> GPFS cluster name: sdx-gpfs.xxxxxxxxxxxxxxxx<br> GPFS cluster id: 12880500218013865782<br> GPFS UID domain: sdx-gpfs. xxxxxxxxxxxxxxxx<br> Remote shell command: /usr/bin/ssh<br> Remote file copy command: /usr/bin/scp<br> Repository type: CCR<br><br> Node Daemon node name IP address
Admin node name Designation<br>---------------------------------------------------------------------------------------<br> 1 mc-5. xxxxxxxxxxxxxxxx.net ip.addresses.hidden mc-5.hidden.net
quorum-manager<br> 2 mc-6. xxxxxxxxxxxxxxxx.net ip.addresses.hidden mc-6.
hidden.net quorum-manager-gateway<br> 3 mc-7. xxxxxxxxxxxxxxxx.net ip.addresses.hidden mc-7.
hidden.net quorum-manager-gateway<br> 4 mc-8. xxxxxxxxxxxxxxxx.net ip.addresses.hidden mc-8.
hidden.net quorum-manager-gateway<br><br>The bit I really don?t get is:<br><br><br>1. Why no traffic ever seems to go through mc-6 or
mc-8 back to my ?home? directly and<br><br>2. Why it only ever lists my AFM-cache fileset being
associated with one gateway (mc-7).<br><br>I can see traffic flowing through mc-6 sometimes?but when it does, it all
seems to channel back through mc-7 THEN back to the AFM-home. Am I missing
something?<br><br>This is where I see one of the gateway?s listed (but never the others?).<br><br>[root@mc-5 ~]# mmafmctl afmcachefs getstate<br>Fileset Name Fileset Target
Cache
State Gateway Node Queue
Length Queue numExec<br>------------ --------------
-------------
------------ ------------
-------------<br>afm-home nfs://omnipath2/gpfs-flash/afm-home
Active
mc-7 0
746636<br><br>I got told I needed to setup ?explicit maps? back to my home cluster to
achieve parallelism:<br><br>[root@mc-5 ~]# mmafmconfig show<br>Map name: omnipath1<br>Export server map: address.is.hidden.100/mc-6.ip.address.hidden<br><br>Map name: omnipath2<br>Export server map: address.is.hidden.101/mc-7.ip.address.hidden<br><br>But ? I have never seen any traffic come back from mc-6 to omnipath1.<br><br>What am I missing, and how do I actually achieve significant enough parallelism
over an NFS transport to fill my 10GbE pipe?<br><br>I?ve seen maybe a couple of gigabits per second from the mc-7 host writing
back to the omnipath2 host ? and that was really trying my level best to
put as many files onto the afm-cache at this side and hoping that enough
threads pick up enough different files to start transferring files down
the AFM simultaneously ? but what I?d really like is those large files
(or small, up to the thresholds set) to break into parallel chunks and
ALL transfer as fast as possible, utilising as much of the 10GbE as they
can.<br><br>Maybe I am missing fundamental principles in the way AFM works?<br><br>Thanks.<br><br>-jc</font></tt><BR>