[gpfsug-discuss] Problem using group pool for migration

Powell (US), Richard E Richard.E.Powell at boeing.com
Wed Aug 22 21:17:44 BST 2018


Allow me to elaborate on my question.  The example I gave was trimmed-down to the minimum.  I've been trying various combinations with different LIMIT values and different weight and where clauses, using '-I test' and '-I prepare' to see what it would do, but not actually doing the migration.  The 'ssd' pool is about 36% utilized and I've been starting the mmapplypolicy scan at a sub-directory level where nearly all the files were in the disk pool.  (You'll just have to trust me that the ssd pool can hold all of them :-))

If I specify 'ssd' as the "to pool", the output from the test or prepare options indicates that it would be able to migrate all of the candidate files to the ssd pool.  But, if I specify the group pool as the "to pool", it is only willing to migrate the first candidate.  That is with the ssd pool listed first in the group and with any limit as long as it's big enough to hold the current data plus the files I expected it to select, even the default of 99.

I'm sure I'm either doing something wrong, or I *really* misunderstand the concept.  It seems straight forward enough....

Thanks to everyone for your time!
Richard


-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of gpfsug-discuss-request at spectrumscale.org
Sent: Wednesday, August 22, 2018 4:00 AM
To: gpfsug-discuss at spectrumscale.org
Subject: gpfsug-discuss Digest, Vol 79, Issue 47

Send gpfsug-discuss mailing list submissions to
	gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
	gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
	gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of gpfsug-discuss digest..."


Today's Topics:

   1. Problem using group pool for migration (Powell (US), Richard E)
   2. Re: Problem using group pool for migration (Marc A Kaplan)
   3. Re: Problem using group pool for migration (Marc A Kaplan)


----------------------------------------------------------------------

Message: 1
Date: Tue, 21 Aug 2018 18:23:50 +0000
From: "Powell (US), Richard E" <Richard.E.Powell at boeing.com>
To: "gpfsug-discuss at spectrumscale.org"
	<gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] Problem using group pool for migration
Message-ID:
	<7a0a914601594ccdb6c96504322de9c8 at XCH15-09-11.nw.nos.boeing.com>
Content-Type: text/plain; charset="us-ascii"

Hi all,

I'm trying to use the "GROUP POOL" feature for file migration with FILE_HEAT, similar to one of the ilm sample scripts.  The problem I'm having is that it seems to be identifying the candidates correctly but, anytime I use the "group pool" name for the "to pool", it only selects the first candidate for migration.  If I specify a single pool name for the "to pool", it selects multiple files as expected.

Here are the policy rules I'm using:

RULE 'gp' GROUP POOL 'gpool' is 'ssd' then 'disk1'
RULE 'repack' MIGRATE FROM POOL 'gpool' TO POOL 'gpool' WEIGHT(FILE_HEAT)

I'm not sure if I'm misunderstanding something or if this is a real bug.  I'm just wondering if anyone else has run into this issue?

I'm running 4.2.3.8 on RHEL 6.

Thanks!
Richard


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180821/b61df7d0/attachment-0001.html>

------------------------------

Message: 2
Date: Tue, 21 Aug 2018 15:45:10 -0400
From: "Marc A Kaplan" <makaplan at us.ibm.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Problem using group pool for migration
Message-ID:
	<OF5A0D3BC1.3631960B-ON852582F0.006B3EE7-852582F0.006C8252 at notes.na.collabserv.com>
	
Content-Type: text/plain; charset="utf-8"

Migrate to a group pool "repacks" the selected files over the pools that comprise the group IN THE ORDER SPECIFIED UP TO THE SPECIFIED LIMIT for each pool.  To see this work, in your case, set a limit that is near the current occupancy of pool 'ssd'.

For example: RULE ?gp? GROUP POOL ?gpool? is ?ssd?  LIMIT(50) then ?disk1?

Notice the documentation says the LIMIT defaults to 99.

Also, if you've run the same policy before and nothings changed much, then of course, there's not going to be much "repacking" to be done, maybe not any.

If the behaviour still doesn't make sense to you, try testing on a tiny file system with just a few small pools, sizing pools and files so that only a few files will fit in a pool...  If you build such a test scenario and that still doesn't make sense, show us the example...

-----------------------------------
From:   "Powell (US), Richard E" <Richard.E.Powell at boeing.com>


Hi all,
 
I?m trying to use the ?GROUP POOL? feature for file migration with FILE_HEAT, similar to one of the ilm sample scripts.  The problem I?m having is that it seems to be identifying the candidates correctly but, anytime I use the ?group pool? name for the ?to pool?, it only selects the first candidate for migration.  If I specify a single pool name for the ?to pool?, it selects multiple files as expected.
 
Here are the policy rules I?m using:
 
RULE ?gp? GROUP POOL ?gpool? is ?ssd? then ?disk1?
RULE ?repack? MIGRATE FROM POOL ?gpool? TO POOL ?gpool? WEIGHT(FILE_HEAT)
 
I?m not sure if I?m misunderstanding something or if this is a real bug. 
I?m just wondering if anyone else has run into this issue?
 
I?m running 4.2.3.8 on RHEL 6.
 
Thanks!
Richard


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180821/58565d79/attachment-0001.html>

------------------------------

Message: 3
Date: Tue, 21 Aug 2018 16:11:10 -0400
From: "Marc A Kaplan" <makaplan at us.ibm.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Problem using group pool for migration
Message-ID:
	<OF26133F2A.9E5B4451-ON852582F0.006E6936-852582F0.006EE3CB at notes.na.collabserv.com>
	
Content-Type: text/plain; charset="us-ascii"

To repack in random order, which might be an interesting and easy way to test and demonstrate... 

Use the RAND() function:

RULE ... MIGRATE ... WEIGHT(RAND()) ... 

-L 3 on the mmapplypolicy command will make the random weights evident in the output.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180821/7bf11e97/attachment-0001.html>

------------------------------

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


End of gpfsug-discuss Digest, Vol 79, Issue 47
**********************************************



More information about the gpfsug-discuss mailing list