<div dir="ltr">Hi,<div><br></div><div>you still need both of them, but they are both on the list to be removed, the first is already integrated for the next major release, the 2nd we still work on. <div><br></div><div>Sven</div><div><br></div><div><div class="gmail_quote"><div dir="ltr">On Wed, Sep 6, 2017 at 4:55 AM Kenneth Waegeman <<a href="mailto:kenneth.waegeman@ugent.be">kenneth.waegeman@ugent.be</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hi Sven,</p>
<p>I see two parameters that we have set to non-default values that
are not in your list of options still to configure.<br>
</p>
verbsRdmasPerConnection (256) and<br>
socketMaxListenConnections (1024)<br>
<br>
I remember we had to set socketMaxListenConnections because our
cluster consist of +550 nodes.<br>
<br>
Are these settings still needed, or is this also tackled in the
code?<br>
<br>
Thank you!!<br>
<br>
Cheers,<br>
Kenneth</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<br>
<div class="m_-5085545595762185036moz-cite-prefix">On 02/09/17 00:42, Sven Oehme wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Ed,
<div><br>
</div>
<div>yes the defaults for that have changed for customers who
had not overridden the default settings. the reason we did
this was that many systems in the field including all ESS
systems that come pre-tuned where manually changed to 8k from
the 16k default due to better performance that was confirmed
in multiple customer engagements and tests with various
settings , therefore we change the default to what it should
be in the field so people are not bothered to set it anymore
(simplification) or get benefits by changing the default to
provides better performance. </div>
<div>all this happened when we did the communication code
overhaul that did lead to significant (think factors) of
improved RPC performance for RDMA and VERBS workloads. </div>
<div>there is another round of significant enhancements coming
soon , that will make even more parameters either obsolete or
change some of the defaults for better out of the box
performance.</div>
<div>i see that we should probably enhance the communication of
this changes, not that i think this will have any negative
effect compared to what your performance was with the old
setting i am actually pretty confident that you get better
performance with the new code, but by setting parameters back
to default on most 'manual tuned' probably makes your system
even faster. </div>
<div>if you have a Scale Client on 4.2.3+ you really shouldn't
have anything set beside maxfilestocache, pagepool,
workerthreads and potential prefetch , if you are a protocol
node, this and settings specific to an export (e.g. SMB, NFS
set some special settings) , pretty much everything else these
days should be set to default so the code can pick the correct
parameters., if its not and you get better performance by
manual tweaking something i like to hear about it.</div>
<div>on the communication side in the next release will
eliminate another set of parameters that are now 'auto set'
and we plan to work on NSD next. </div>
<div>i presented various slides about the communication and
simplicity changes in various forums, latest public non NDA
slides i presented are here --> <a href="http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf" target="_blank">http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf</a></div>
<div><br>
</div>
<div>hope this helps . </div>
<div><br>
</div>
<div>Sven</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Fri, Sep 1, 2017 at 1:56 PM Edward Wahl <<a href="mailto:ewahl@osc.edu" target="_blank"></a><a class="m_-5085545595762185036moz-txt-link-abbreviated" href="mailto:ewahl@osc.edu" target="_blank">ewahl@osc.edu</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Howdy.
Just noticed this change to min RDMA packet size and I don't
seem to<br>
see it in any patch notes. Maybe I just skipped the one where
this changed?<br>
<br>
mmlsconfig verbsRdmaMinBytes<br>
verbsRdmaMinBytes 16384<br>
<br>
(in case someone thinks we changed it)<br>
<br>
[root@proj-nsd01 ~]# mmlsconfig |grep verbs<br>
verbsRdma enable<br>
verbsRdma disable<br>
verbsRdmasPerConnection 14<br>
verbsRdmasPerNode 1024<br>
verbsPorts mlx5_3/1<br>
verbsPorts mlx4_0<br>
verbsPorts mlx5_0<br>
verbsPorts mlx5_0 mlx5_1<br>
verbsPorts mlx4_1/1<br>
verbsPorts mlx4_1/2<br>
<br>
<br>
Oddly I also see this in config, though I've seen these kinds
of things before.<br>
mmdiag --config |grep verbsRdmaMinBytes<br>
verbsRdmaMinBytes 8192<br>
<br>
We're on a recent efix.<br>
Current GPFS build: "4.2.2.3 efix21 (1028007)".<br>
<br>
--<br>
<br>
Ed Wahl<br>
Ohio Supercomputer Center<br>
<a href="tel:%28614%29%20292-9302" value="+16142929302" target="_blank">614-292-9302</a><br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote>
</div>
<br>
<fieldset class="m_-5085545595762185036mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a>
<a class="m_-5085545595762185036moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div></div></div></div>