[gpfsug-discuss] Mmapplypolicy with -I defer doesn't sort resulting list? Is this by intention?

[IBM Spectrum Scale]

Hi Billich,

I think the problem is that you are specifying --choice-algorithm fast and
as per documentation "The fast choice method does not completely sort the
candidates by weight."
To sort the list you can try specifying --choice-algorithm exact which is
also the default.


Regards, The Spectrum Scale (GPFS) team

------------------------------------------------------------------------------------------------------------------

If you feel that your question can benefit other users of  Spectrum Scale
(GPFS), then please post it to the public IBM developerWroks Forum at
https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479.


If your query concerns a potential software error in Spectrum Scale (GPFS)
and you have an IBM software maintenance contract please contact
1-800-237-5511 in the United States or your local IBM Service Center in
other countries.

The forum is informally monitored as time permits and should not be used
for priority messages to the Spectrum Scale (GPFS) team.



From:	"Billich  Heinrich Rainer (ID SD)"
            <heinrich.billich at id.ethz.ch>
To:	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:	09-02-2021 03.13 AM
Subject:	[EXTERNAL] [gpfsug-discuss] Mmapplypolicy with -I defer doesn't
            sort resulting list? Is this by intention?
Sent by:	gpfsug-discuss-bounces at spectrumscale.org



Hello,

I want to migrate data with mmapplypolicy to a different pool:

1. create a file list with "-I defer -f /some/path"
2. execute with "-I yes -r /some/path"

I noted that the file list created in 1. Is not sorted. I asked to sort by
kb_allocated, the ideas is to migrate the largest files first and to stop
once we freed enough space. Is this intended, or should the file list
created in 1. be sorted? I want to be able to review the list before I
start the migration.

The list holds size/kb_allocated  in hex numbers, I didn't manage to sort
it with 'sort'. Hence I would prefer to get a sorted list from
mmapplypolicy instead of writing some code to do this.

The list holds files with sizes between 32K and 250G and 4.5M lines, hence
it makes sense to sort to move the largest files on the top.

Thank you and best regards,

Heiner



The policy is

RULE 'migrate'
 MIGRATE
   FROM POOL 'data'
   THRESHOLD (50,50)
   WEIGHT (kb_allocated)
 TO POOL 'capacity'
   LIMIT (98)
 WHERE kb_allocated > 0 AND (DAYS(CURRENT_TIMESTAMP) - DAYS(ACCESS_TIME)) >
90

And the command line

mmapplypolicy $1 -A 600 \
              -a 10 \
              -P $base/policy.txt \
              -I prepare \
              -f $base/candidates \
              -L 1 \
              -N some_nodes \
              --choice-algorithm fast \
              --max-sort-bytes 1G \
              --split-margin 2.0

The filesystem holds about 1.2G inodes. I tried to speed-up sorting with
the last three arguments.
--
=======================
Heinrich Billich
ETH Zürich
Informatikdienste
Tel.: +41 44 632 72 56
heinrich.billich at id.ethz.ch
========================



[attachment "smime.p7s" deleted by Huzefa H Pancha/India/IBM]
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210209/9c45e78f/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210209/9c45e78f/attachment-0002.gif>