[gpfsug-discuss] restripe or not

[Stijn De Weirdt]

hi bob,

> - One thing you might consider (depending on your pattern of read/write
> traffic), is selectively suspending one or more of the existing NSDs,
> forcing GPFS to write new blocks to the new NSDs. That way at least some of
> the new data is being written to the new storage by default, rather than
> using up blocks on the existing NSDs. You can suspend/resume disks at any
> time.
is the gpfs placment weighted with the avalaible volume? i'd rather not 
make this a manual operation.

>
> - You can pick a subset of nodes to perform the restripe with "mmrestripefs
> -N node1,node2,..." Keep in mind you'll get much better performance and
> less impact to the filesystem if you choose NSD servers with direct access
> to the disk.
yes and i no i guess, our nsds see all disks, but the problem with nsds 
is that they don't honour any roles (our primary nsds have the preferred 
path to the controller and lun, meaning all access from non-primary nsd 
to that disk is suboptimal).

>
> - Resume of restripe: Yes, you can do this, no harm, done it many times.
> You can track the balance of the disks using "mmdf <filesystem>". This is a
> pretty intensive command, so I wouldn't run in frequently. Check it a few
> times each day, see if the data balance is improving by itself. When you
thanks for the tip to monitor it with mmdf!
> stop/restart it, the restripe doesn't pick up exactly where it left off,
> it's going to scan the entire file system again.
yeah, i realised that this is a flaw in my "one-hour a day" restripe idea ;)

>
> - You can also restripe single files if the are large and get a heavy I/O
> (mmrestripefile)
excellent tip! forgot about that one. if the rebalnce is to slow, i can 
run this on the static data.

thanks a lot for the feedback

stijn

>
> Bob Oesterlin
> Nuance Communications
>
>
> On Mon, Nov 24, 2014 at 3:22 PM, Stijn De Weirdt <stijn.deweirdt at ugent.be>
> wrote:
>
>> hi all,
>>
>> we are going to expand an existing filestytem with approx 50% capacity.
>> the current filesystem is 75% full.
>>
>> we are in downtime (for more then just this reason), so we can take the IO
>> rebalance hit for a while (say max 48hours).
>>
>> my questions:
>> a. do we really need to rebalance? the mmadddisk page suggest normally
>> it's going to be ok, but i never understood that. new data will end up
>> mainly on new disks, so wrt to performance, this can't really work out, can
>> it?
>> b. can we change the priority of rebalancing somehow (fewer nodes taking
>> part in the rebalance?)
>> c. once we start the rebalance, how save is it to stop with kill or ctrl-c
>> (or can we say eg. rebalance 25% now, rest later?)
>> (and how often can we do this? eg a daily cron job to restripe at max one
>> hour per day, would this cause issue in the long term
>>
>>
>> many thanks,
>>
>> stijn
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at gpfsug.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>