[gpfsug-discuss] Rebalancing with mmrestripefs -P

[david_johnson at brown.edu]

Yes the arrays are in different buildings. We want to spread the activity over more servers if possible but recognize the extra load that rebalancing would entail. The system is busy all the time. 

I have considered using QOS when we run policy migrations but haven’t yet because I don’t know what value to allow for throttling IOPS.  We need to do weekly migrations off of 15k rpm pool onto 7.2k rpm pool, and previously I’ve just let it run at native speed. I’d like to know what other folks have used for QOS settings. 

I think we may leave things alone for now regarding the original question, rebalancing this pool. 

  -- ddj
Dave Johnson

> On Aug 20, 2018, at 6:08 PM, valdis.kletnieks at vt.edu wrote:
> 
> On Mon, 20 Aug 2018 14:02:05 -0400, "Frederick Stock" said:
> 
>> Note you have two additional NSDs in the 33 failure group than you do in
>> the 23 failure group.  You may want to change one of those NSDs in failure
>> group 33 to be in failure group 23 so you have equal storage space in both
>> failure groups.
> 
> Keep in mind that the failure groups should be built up based on single points of failure.
> In other words, a failure group should consist of disks that will all stay up or all go down on
> the same failure (controller, network, whatever).
> 
> Looking at the fact that you have 6 disks named 'dNN_george_33' and  8 named 'dNN_cit_33',
> it sounds very likely that they are in two different storage arrays, and you should make your
> failure groups so they don't span a storage array. In other words, taking a 'cit' disk
> and moving it into a 'george' failure group will Do The Wrong Thing, because if you do
> data replication, one copy can go onto a 'george' disk, and the other onto a 'cit' disk
> that's in the same array as the 'george' disk.  If 'george' fails, you lose access to both
> replicas.
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss