<html><body><p><font size="2">The discrepancy between the mmlsconfig view and mmdiag has been fixed in GFPS 4.2.3 version. Note, mmdiag reports the correct default value.</font><br><br><font size="2">Tru.</font><br><br><img width="16" height="16" src="cid:1__=0ABB0B1DDFEE41268f9e8a93df938690918c0AB@" border="0" alt="Inactive hide details for gpfsug-discuss-request---09/01/2017 06:43:20 PM---Send gpfsug-discuss mailing list submissions to gp"><font size="2" color="#424282">gpfsug-discuss-request---09/01/2017 06:43:20 PM---Send gpfsug-discuss mailing list submissions to gpfsug-discuss@spectrumscale.org</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">gpfsug-discuss-request@spectrumscale.org</font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug-discuss@spectrumscale.org</font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">09/01/2017 06:43 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">gpfsug-discuss Digest, Vol 68, Issue 2</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt><font size="2">Send gpfsug-discuss mailing list submissions to<br> gpfsug-discuss@spectrumscale.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br> </font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=</a></font></tt><tt><font size="2"> <br>or, via email, send a message with subject or body 'help' to<br> gpfsug-discuss-request@spectrumscale.org<br><br>You can reach the person managing the list at<br> gpfsug-discuss-owner@spectrumscale.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of gpfsug-discuss digest..."<br><br><br>Today's Topics:<br><br> 1. Re: GPFS GUI Nodes > NSD no data (Sobey, Richard A)<br> 2. Change to default for verbsRdmaMinBytes? (Edward Wahl)<br> 3. Quorum managers (Joshua Akers)<br> 4. Re: Change to default for verbsRdmaMinBytes? (Sven Oehme)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Fri, 1 Sep 2017 13:36:56 +0000<br>From: "Sobey, Richard A" <r.sobey@imperial.ac.uk><br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Subject: Re: [gpfsug-discuss] GPFS GUI Nodes > NSD no data<br>Message-ID:<br> <HE1PR0602MB3225B01281871606C60C6333DF920@HE1PR0602MB3225.eurprd06.prod.outlook.com><br> <br>Content-Type: text/plain; charset="us-ascii"<br><br>Resolved this, guessed at changing GPFSNSDDisk.period to 5.<br><br>From: gpfsug-discuss-bounces@spectrumscale.org [</font></tt><tt><font size="2"><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a></font></tt><tt><font size="2">] On Behalf Of Sobey, Richard A<br>Sent: 01 September 2017 09:45<br>To: 'gpfsug-discuss@spectrumscale.org' <gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] GPFS GUI Nodes > NSD no data<br><br>For some time now if I go into the GUI, select Monitoring > Nodes > NSD Server Nodes, the only columns with good data are Name, State and NSD Count. Everything else e.g. Avg Disk Wait Read is listed "N/A".<br><br>Is this another config option I need to enable? It's been bugging me for a while, I don't think I've seen it work since 4.2.1 which was the first time I saw the GUI.<br><br>Cheers<br>Richard<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_2a4162e9_attachment-2D0001.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=jcPGl5zwtQFMbnEmBpNErsD43uwoVeKgKk_8j7ZeCJY&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_2a4162e9_attachment-2D0001.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=jcPGl5zwtQFMbnEmBpNErsD43uwoVeKgKk_8j7ZeCJY&e=</a></font></tt><tt><font size="2"> ><br><br>------------------------------<br><br>Message: 2<br>Date: Fri, 1 Sep 2017 16:56:25 -0400<br>From: Edward Wahl <ewahl@osc.edu><br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] Change to default for verbsRdmaMinBytes?<br>Message-ID: <20170901165625.6e4edd4c@osc.edu><br>Content-Type: text/plain; charset="US-ASCII"<br><br>Howdy. Just noticed this change to min RDMA packet size and I don't seem to<br>see it in any patch notes. Maybe I just skipped the one where this changed?<br><br> mmlsconfig verbsRdmaMinBytes<br>verbsRdmaMinBytes 16384 <br><br>(in case someone thinks we changed it)<br><br>[root@proj-nsd01 ~]# mmlsconfig |grep verbs<br>verbsRdma enable<br>verbsRdma disable<br>verbsRdmasPerConnection 14<br>verbsRdmasPerNode 1024<br>verbsPorts mlx5_3/1<br>verbsPorts mlx4_0<br>verbsPorts mlx5_0<br>verbsPorts mlx5_0 mlx5_1<br>verbsPorts mlx4_1/1<br>verbsPorts mlx4_1/2<br><br><br>Oddly I also see this in config, though I've seen these kinds of things before.<br>mmdiag --config |grep verbsRdmaMinBytes<br> verbsRdmaMinBytes 8192<br><br>We're on a recent efix. <br>Current GPFS build: "4.2.2.3 efix21 (1028007)".<br><br>-- <br><br>Ed Wahl<br>Ohio Supercomputer Center<br>614-292-9302<br><br><br>------------------------------<br><br>Message: 3<br>Date: Fri, 01 Sep 2017 21:06:15 +0000<br>From: Joshua Akers <akers@vt.edu><br>To: "gpfsug-discuss@spectrumscale.org"<br> <gpfsug-discuss@spectrumscale.org><br>Subject: [gpfsug-discuss] Quorum managers<br>Message-ID:<br> <CAHO5rBG+PkntpshV105j54+O4CtcDXqQCb9AJutq-s_PEN0g3A@mail.gmail.com><br>Content-Type: text/plain; charset="utf-8"<br><br>Hi all,<br><br>I was wondering how most people set up quorum managers. We historically had<br>physical admin nodes be the quorum managers, but are switching to a<br>virtualized admin services infrastructure. We have been choosing a few<br>compute nodes to act as quorum managers in our client clusters, but have<br>considered using virtual machines instead. Has anyone else done this?<br><br>Regards,<br>Josh<br>-- <br>*Joshua D. Akers*<br><br>*HPC Team Lead*<br>NI&S Systems Support (MC0214)<br>1700 Pratt Drive<br>Blacksburg, VA 24061<br>540-231-9506<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_a49947db_attachment-2D0001.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=Gag0raQbp7KZAyINlnmuxlnpjboo9XOWO3dDL2HCsZo&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_a49947db_attachment-2D0001.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=Gag0raQbp7KZAyINlnmuxlnpjboo9XOWO3dDL2HCsZo&e=</a></font></tt><tt><font size="2"> ><br><br>------------------------------<br><br>Message: 4<br>Date: Fri, 01 Sep 2017 22:42:55 +0000<br>From: Sven Oehme <oehmes@gmail.com><br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Subject: Re: [gpfsug-discuss] Change to default for verbsRdmaMinBytes?<br>Message-ID:<br> <CALssuR3-KmWd4LqKZwQkP7_F6+aX3JM-KqRnc=+czkxMP3xCMg@mail.gmail.com><br>Content-Type: text/plain; charset="utf-8"<br><br>Hi Ed,<br><br>yes the defaults for that have changed for customers who had not overridden<br>the default settings. the reason we did this was that many systems in the<br>field including all ESS systems that come pre-tuned where manually changed<br>to 8k from the 16k default due to better performance that was confirmed in<br>multiple customer engagements and tests with various settings , therefore<br>we change the default to what it should be in the field so people are not<br>bothered to set it anymore (simplification) or get benefits by changing the<br>default to provides better performance.<br>all this happened when we did the communication code overhaul that did lead<br>to significant (think factors) of improved RPC performance for RDMA and<br>VERBS workloads.<br>there is another round of significant enhancements coming soon , that will<br>make even more parameters either obsolete or change some of the defaults<br>for better out of the box performance.<br>i see that we should probably enhance the communication of this changes,<br>not that i think this will have any negative effect compared to what your<br>performance was with the old setting i am actually pretty confident that<br>you get better performance with the new code, but by setting parameters<br>back to default on most 'manual tuned' probably makes your system even<br>faster.<br>if you have a Scale Client on 4.2.3+ you really shouldn't have anything set<br>beside maxfilestocache, pagepool, workerthreads and potential prefetch , if<br>you are a protocol node, this and settings specific to an export (e.g.<br>SMB, NFS set some special settings) , pretty much everything else these<br>days should be set to default so the code can pick the correct parameters.,<br>if its not and you get better performance by manual tweaking something i<br>like to hear about it.<br>on the communication side in the next release will eliminate another set of<br>parameters that are now 'auto set' and we plan to work on NSD next.<br>i presented various slides about the communication and simplicity changes<br>in various forums, latest public non NDA slides i presented are here --><br></font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=8c_55Ld_iAC2sr_QU0cyGiOiyU7Z9NjcVknVuRpRIlk&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=8c_55Ld_iAC2sr_QU0cyGiOiyU7Z9NjcVknVuRpRIlk&e=</a></font></tt><tt><font size="2"> <br><br>hope this helps .<br><br>Sven<br><br><br><br>On Fri, Sep 1, 2017 at 1:56 PM Edward Wahl <ewahl@osc.edu> wrote:<br><br>> Howdy. Just noticed this change to min RDMA packet size and I don't seem<br>> to<br>> see it in any patch notes. Maybe I just skipped the one where this<br>> changed?<br>><br>> mmlsconfig verbsRdmaMinBytes<br>> verbsRdmaMinBytes 16384<br>><br>> (in case someone thinks we changed it)<br>><br>> [root@proj-nsd01 ~]# mmlsconfig |grep verbs<br>> verbsRdma enable<br>> verbsRdma disable<br>> verbsRdmasPerConnection 14<br>> verbsRdmasPerNode 1024<br>> verbsPorts mlx5_3/1<br>> verbsPorts mlx4_0<br>> verbsPorts mlx5_0<br>> verbsPorts mlx5_0 mlx5_1<br>> verbsPorts mlx4_1/1<br>> verbsPorts mlx4_1/2<br>><br>><br>> Oddly I also see this in config, though I've seen these kinds of things<br>> before.<br>> mmdiag --config |grep verbsRdmaMinBytes<br>> verbsRdmaMinBytes 8192<br>><br>> We're on a recent efix.<br>> Current GPFS build: "4.2.2.3 efix21 (1028007)".<br>><br>> --<br>><br>> Ed Wahl<br>> Ohio Supercomputer Center<br>> 614-292-9302 <(614)%20292-9302><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=</a></font></tt><tt><font size="2"> <br>><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_b75cfc74_attachment.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=LpVpXMgqE_LD-t_J7yfNwURUrdUR29TzWvjVTi18kpA&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20170901_b75cfc74_attachment.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=LpVpXMgqE_LD-t_J7yfNwURUrdUR29TzWvjVTi18kpA&e=</a></font></tt><tt><font size="2"> ><br><br>------------------------------<br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HQmkdQWQHoc1Nu6Mg_g8NVugim3OiUUy5n0QgLQcbkM&m=yK4FkYvJ21ubvurR6W1Pi3qvNw9ydj2XP0ghXPc7DUw&s=xZHUN9ZlFjvgBmBB8wnX2cQDQQV42R_q-xHubNA3JBM&e=</a></font></tt><tt><font size="2"> <br><br><br>End of gpfsug-discuss Digest, Vol 68, Issue 2<br>*********************************************<br><br></font></tt><br><br><BR>
</body></html>