<span style=" font-size:10pt;font-family:sans-serif">Hi</span><br><br><span style=" font-size:10pt;font-family:sans-serif"><br>What is the output for:<br>#mmlsconfig |grep -i verbs </span><br><span style=" font-size:10pt;font-family:sans-serif">#ibstat<br></span><br><span style=" font-size:9pt;font-family:Arial"> </span><br><span style=" font-size:10pt;font-family:Arial">Regards</span><br><span style=" font-size:9pt;font-family:Arial"> </span><br><table width=780 style="border-collapse:collapse;"><tr height=8><td width=780 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center><hr noshade></div><br><span style=" font-size:1pt;font-family:Arial"> </span><tr height=8><td width=780 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:1pt;font-family:Arial"> </span><tr height=8><td width=516 colspan=2 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:blue;font-family:Arial"><b>Yaron

Daniel</b></span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> 94

Em Ha'Moshavot Rd</span><td width=96 rowspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=right><img align=bottom src=cid:_1_08807E58088078CC00274C4DC22585AE style="border:0px solid;"></div><tr height=8><td width=516 colspan=2 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:blue;font-family:Arial"><b>Storage

Architect – IL Lab Services (Storage)</b></span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> Petach

Tiqva, 49527</span><tr height=8><td width=516 colspan=2 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:blue;font-family:Arial"><b>IBM

Global Markets, Systems HW Sales</b></span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> Israel</span><tr height=8><td width=516 colspan=2 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:blue;font-family:Arial"><b> </b></span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> </span><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:9pt;font-family:Arial"> </span><tr height=8><td width=90 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">Phone:</span><td width=426 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">+972-3-916-5672</span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> </span><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt"> </span><tr height=8><td width=90 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">Fax:</span><td width=426 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">+972-3-916-5672</span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> 

</span><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt"> </span><tr height=8><td width=90 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">Mobile:</span><td width=426 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">+972-52-8395593</span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> 

</span><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt"> </span><tr height=8><td width=90 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">e-mail:</span><td width=426 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial">yard@il.ibm.com</span><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;color:#5f5f5f;font-family:Arial"> 

</span><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt"> </span><tr height=8><td width=516 colspan=2 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><a href="Webex:%20%20%20%20%20%20%20%20%20%20%20%20%20https://ibm.webex.com/meet/yard%0dIBM%20Israel"><span style=" font-size:12pt;color:#5f5f5f;font-family:Times New Roman">Webex:           

https://ibm.webex.com/meet/yard</span></a><span style=" font-size:10pt;color:blue;font-family:Arial"><u><br></u></span><a href="Webex:%20%20%20%20%20%20%20%20%20%20%20%20%20https://ibm.webex.com/meet/yard%0dIBM%20Israel"><span style=" font-size:10pt;color:blue;font-family:Arial"><u>IBM

Israel</a><td width=168 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><td width=96 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt"> <tr height=8><td width=780 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:9pt;color:#5f5f5f;font-family:Arial"> <tr height=8><td width=780 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:9pt;color:#5f5f5f;font-family:Arial"> </table><p style="margin-top:0px;margin-Bottom:0px"> <span style=" font-size:12pt">  <img src=cid:_1_0E57D6F80E57D2E000274C4DC22585AE alt="IBM Storage for Cyber Resiliency & Modern Data Protection V1" style="border:0px solid;"><img src=cid:_2_0E57D9040E57D2E000274C4DC22585AE style="border:0px solid;"><img src=cid:_2_0E57DB100E57D2E000274C4DC22585AE style="border:0px solid;"><img src=cid:_2_0E57DD1C0E57D2E000274C4DC22585AE style="border:0px solid;"><img src=cid:_2_0E57DF280E57D2E000274C4DC22585AE style="border:0px solid;"> <img src=cid:_2_0E57E5AC0E57E19400274C4DC22585AE style="border:0px solid;"><img src=cid:_2_0E58E4480E57E19400274C4DC22585AE style="border:0px solid;"><img src=cid:_2_0E58E6540E57E19400274C4DC22585AE style="border:0px solid;"><span style=" font-size:12pt"> <img src=cid:_1_0E58E8600E57E19400274C4DC22585AE alt="IBM Storage and Cloud Essentials" style="border:0px solid;"><img src=cid:_2_0E58EA6C0E57E19400274C4DC22585AE style="border:0px solid;"> <span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:

       </span><span style=" font-size:9pt;font-family:sans-serif">Prasad

Surampudi <prasad.surampudi@theatsgroup.com> <span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:

       </span><span style=" font-size:9pt;font-family:sans-serif">"gpfsug-discuss@spectrumscale.org"

<gpfsug-discuss@spectrumscale.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:

       </span><span style=" font-size:9pt;font-family:sans-serif">07/23/2020

03:34 AM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:

       </span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]

[gpfsug-discuss] Spectrum Scale pagepool size with RDMA</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Sent

by:        </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><br><hr noshade><br><br><br><span style=" font-size:12pt;font-family:Calibri">Hi,</span><br><br><span style=" font-size:12pt;font-family:Calibri">We have an ESS clusters

with two CES nodes. The pagepool is set to 128 GB ( Real Memory is 256

GB ) on both ESS NSD servers and CES nodes as well. Occasionally we see

the mmfsd process memory usage reaches 90% on NSD servers and CES nodes

and stays there until GPFS is recycled. I have couple of questions in this

scenario:</span><br><ol><li value=1><span style=" font-size:12pt;font-family:Calibri"> What

are the general recommendations of pagepool size for nodes with RDMA enabled?

On, IBM knowledge center for RDMA tuning says "</span><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial">If

the GPFS pagepool is set to 32 GB, then the mapping of the RDMA for this

pagepool must be at least 64 GB."  So, does this mean that the

pagepool can't be more than half of real memory with RDMA enabled? Also,

Is this the reason why mmfsd memory usage exceeds pagepool size and spikes

to almost 90%?</span><li value=2><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial">If

we dont want to see high mmfsd process memory usage on NSD/CES nodes, should

we decrease the pagepool size?</span><li value=3><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial">Can

we tune  </span><span style=" font-size:12pt;color:#2f2f2f;font-family:Courier">log_num_mtt

parameter to limit the memory usage? Currently its set to 0 for both NSD

(ppc64_le) and CES (x86_64).</span><li value=4><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial">We

also see messages like "Verbs RDMA disabled for xx.xx.xx.xx due to

no matching port found" . Any idea what this message indicate? I dont

see any Verbs RDMA enabled message after these warning messages. Does it

get enabled automatically? </span></ol><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:11pt;color:#2020a0;font-family:Tahoma"><b>Prasad

Surampudi<span style=" font-size:9pt;color:#ff8141;font-family:Tahoma">|<span style=" font-size:9pt;color:#000080;font-family:Tahoma">Sr.

Systems Engineer</span><span style=" font-size:9pt;color:#ff8100;font-family:Tahoma">|</span><a href="http://www.theatsgroup.com/" target="_blank"><span style=" font-size:9pt;color:#0062e1;font-family:Tahoma"><u>ATS

Group, LLC</u></span></a></p><p style="margin-top:0px;margin-Bottom:0px"></p><br><br><tt><span style=" font-size:10pt">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br></span></tt><br><br><BR>