<div dir="ltr">Hey Dave,<div><br></div><div>Can you say more about what you are trying to accomplish by doing the rebalance? IME, the performance hit from running the rebalance was higher than the performance hit from writes being directed to a subset of the disks.</div><div><br></div><div>If you have any churn of the data, eventually they will rebalance anyway.</div><div><br></div><div>Regards,</div><div>Alex</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Aug 20, 2018 at 11:06 AM <<a href="mailto:david_johnson@brown.edu">david_johnson@brown.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Does anyone have a good rule of thumb for iops to allow for background QOS tasks? <div><br></div><div><br><br><div id="m_3766467585282830855AppleMailSignature"> -- ddj<div>Dave Johnson</div></div><div><br>On Aug 20, 2018, at 2:02 PM, Frederick Stock <<a href="mailto:stockf@us.ibm.com" target="_blank">stockf@us.ibm.com</a>> wrote:<br><br></div><blockquote type="cite"><div><font size="3" face="Arial">That should do what you want. Be aware
that mmrestripefs generates significant IO load so you should either use
the QoS feature to mitigate its impact or run the command when the system
is not very busy.</font><br><br><font size="3" face="Arial">Note you have two additional NSDs in the
33 failure group than you do in the 23 failure group. You may want
to change one of those NSDs in failure group 33 to be in failure group
23 so you have equal storage space in both failure groups.</font><br><br><font size="3" face="sans-serif">Fred<br>__________________________________________________<br>Fred Stock | IBM Pittsburgh Lab | 720-430-8821<br><a href="mailto:stockf@us.ibm.com" target="_blank">stockf@us.ibm.com</a></font><br><br><br><br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif">David Johnson <<a href="mailto:david_johnson@brown.edu" target="_blank">david_johnson@brown.edu</a>></font><br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif">gpfsug main discussion
list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>></font><br><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">08/20/2018 12:55 PM</font><br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">[gpfsug-discuss]
Rebalancing with mmrestripefs -P</font><br><font size="1" color="#5f5f5f" face="sans-serif">Sent by:
</font><font size="1" face="sans-serif"><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a></font><br><hr noshade><br><br><br><font size="3">I have one storage pool that was recently doubled, and
another pool migrated there using mmapplypolicy.</font><br><font size="3">The new half is only 50% full, and the old half is 94%
full. </font><br><br><font size="1">Disks in storage pool: cit_10tb (Maximum disk size allowed
is 516 TB)</font><br><font size="1">d05_george_23 50.49T
23 No Yes
25.91T ( 51%) 18.93G ( 0%) </font><br><font size="1">d04_george_23 50.49T
23 No Yes
25.91T ( 51%) 18.9G ( 0%) </font><br><font size="1">d03_george_23 50.49T
23 No Yes
25.9T ( 51%) 19.12G ( 0%) </font><br><font size="1">d02_george_23 50.49T
23 No Yes
25.9T ( 51%) 19.03G ( 0%) </font><br><font size="1">d01_george_23 50.49T
23 No Yes
25.9T ( 51%) 18.92G ( 0%) </font><br><font size="1">d00_george_23 50.49T
23 No Yes
25.91T ( 51%) 19.05G ( 0%) </font><br><font size="1">d06_cit_33 50.49T
33 No Yes
3.084T ( 6%) 70.35G ( 0%)
</font><br><font size="1">d07_cit_33 50.49T
33 No Yes
3.084T ( 6%) 70.2G ( 0%)
</font><br><font size="1">d05_cit_33 50.49T
33 No Yes
3.084T ( 6%) 69.93G ( 0%)
</font><br><font size="1">d04_cit_33 50.49T
33 No Yes
3.085T ( 6%) 70.11G ( 0%)
</font><br><font size="1">d03_cit_33 50.49T
33 No Yes
3.084T ( 6%) 70.08G ( 0%)
</font><br><font size="1">d02_cit_33 50.49T
33 No Yes
3.083T ( 6%) 70.3G ( 0%)
</font><br><font size="1">d01_cit_33 50.49T
33 No Yes
3.085T ( 6%) 70.25G ( 0%)
</font><br><font size="1">d00_cit_33 50.49T
33 No Yes
3.083T ( 6%) 70.28G ( 0%)
</font><br><font size="1">
-------------
-------------------- -------------------</font><br><font size="1">(pool total) 706.9T
180.1T ( 25%) 675.5G
( 0%)</font><br><br><font size="3"> Will the command "mmrestripfs /gpfs -b -P cit_10tb”
move the data blocks from the _cit_ NSDs to the _george_ NSDs,</font><br><font size="3">so that they end up all around 75% full?</font><br><br><font size="3">Thanks,</font><br><font size="3"> — ddj</font><br><font size="3">Dave Johnson</font><br><font size="3">Brown University CCV/CIS</font><tt><font size="2">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br></font></tt><br><br><br>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>gpfsug-discuss mailing list</span><br><span>gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a></span><br><span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span><br></div></blockquote></div></div>_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div>