<html><head></head><body>Like Jan, I did some benchmarking a few years ago when the default recommended RG's dropped to 1 per DA to meet rebuild requirements. I couldn't see any discernable difference.<br><br>As Achim has also mentioned, I just use vdisks for creating additional filesystems. Unless there is going to be a lot of shuffling of space or future filesystem builds, then I divide the RG's into say 10 vdisks to give some flexibility and granularity<br><br>There is also a flag iirc that changes the gpfs magic to consider multiple under lying disks, when I find it again........ Which can provide increased performance on traditional RAID builds.<br><br>-- Lauz<br><br><div class="gmail_quote">On 1 March 2021 08:16:43 GMT, Achim Rehor <Achim.Rehor@de.ibm.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">The reason for having multiple NSDs in legacy NSD (non-GNR) handling is <br>the increased parallelism, that gives you 'more spindles' and thus more <br>performance.<br>In GNR the drives are used in parallel anyway through the GNRstriping. <br>Therfore, you are using all drives of a ESS/GSS/DSS model under the hood <br>in the vdisks anyway. <br><br>The only reason for having more NSDs is for using them for different <br>filesystems. <br><br> <br>Mit freundlichen Grüßen / Kind regards<br><br>Achim Rehor<br><br>IBM EMEA ESS/Spectrum Scale Support<br><br><br><br><br><br><br><br><br><br><br><br><br>gpfsug-discuss-bounces@spectrumscale.org wrote on 01/03/2021 08:58:43:<br><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">From: Jonathan Buzzard <jonathan.buzzard@strath.ac.uk><br>To: gpfsug-discuss@spectrumscale.org<br>Date: 01/03/2021 08:58<br>Subject: [EXTERNAL] Re: [gpfsug-discuss] dssgmkfs.mmvdisk number of <br></blockquote>NSD's<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">Sent by: gpfsug-discuss-bounces@spectrumscale.org<br><br>On 28/02/2021 09:31, Jan-Frode Myklebust wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;"><br>I?ve tried benchmarking many vs. few vdisks per RG, and never could <br></blockquote></blockquote>see <br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">any performance difference.<br></blockquote><br>That's encouraging.<br><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;"><br>Usually we create 1 vdisk per enclosure per RG, thinking this will <br>allow us to grow with same size vdisks when adding additional <br></blockquote></blockquote>enclosures <br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">in the future.<br><br>Don?t think mmvdisk can be told to create multiple vdisks per RG <br>directly, so you have to manually create multiple vdisk sets each with <br></blockquote></blockquote><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">the apropriate size.<br><br></blockquote><br>Thing is back in the day so GPFS v2.x/v3.x there where strict warnings <br>that you needed a minimum of six NSD's for optimal performance. I have <br>sat in presentations where IBM employees have said so. What we where <br>told back then is that GPFS needs a minimum number of NSD's in order to <br>be able to spread the I/O's out. So if an NSD is being pounded for reads <br></blockquote><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">and a write comes in it. can direct it to a less busy NSD.<br><br>Now I can imagine that in a ESS/DSS-G that as it's being scattered to <br>the winds under the hood this is no longer relevant. But some notes to <br>the effect for us old timers would be nice if that is the case to put <br>our minds to rest.<br><br><br>JAB.<br><br>-- <br>Jonathan A. Buzzard Tel: +44141-5483420<br>HPC System Administrator, ARCHIE-WeSt.<br>University of Strathclyde, John Anderson Building, Glasgow. G4 0NG<hr>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="https://urldefense.proofpoint.com/v2/url?">https://urldefense.proofpoint.com/v2/url?</a><br><br></blockquote>u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">siA1ZOg&r=RGTETs2tk0Kz_VOpznDVDkqChhnfLapOTkxLvgmR2-<br>M&m=Mr4A8ROO2t7qFYTfTRM_LoPLllETw72h51FK07dye7Q&s=z6yRHIKsH-<br>IaOjtto4ZyUjFFe0vTGhqzYUiM23rEShg&e= <br><br></blockquote><hr>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br></pre></blockquote></div><br>-- <br>Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>