[gpfsug-discuss] Mmapplypolicy with -I defer doesn't sort resulting list? Is this by intention?

[Billich Heinrich Rainer (ID SD)]

Hello,

I want to migrate data with mmapplypolicy to a different pool:

1. create a file list with "-I defer -f /some/path" 
2. execute with "-I yes -r /some/path"

I noted that the file list created in 1. Is not sorted. I asked to sort by kb_allocated, the ideas is to migrate the largest files first and to stop once we freed enough space. Is this intended, or should the file list created in 1. be sorted? I want to be able to review the list before I start the migration.

The list holds size/kb_allocated  in hex numbers, I didn't manage to sort it with 'sort'. Hence I would prefer to get a sorted list from mmapplypolicy instead of writing some code to do this.

The list holds files with sizes between 32K and 250G and 4.5M lines, hence it makes sense to sort to move the largest files on the top.

Thank you and best regards,

Heiner



The policy is

RULE 'migrate'
 MIGRATE
   FROM POOL 'data'
   THRESHOLD (50,50)
   WEIGHT (kb_allocated)
 TO POOL 'capacity'
   LIMIT (98)
 WHERE kb_allocated > 0 AND (DAYS(CURRENT_TIMESTAMP) - DAYS(ACCESS_TIME)) > 90

And the command line

mmapplypolicy $1 -A 600 \
              -a 10 \
              -P $base/policy.txt \
              -I prepare \
              -f $base/candidates \
              -L 1 \
              -N some_nodes \
              --choice-algorithm fast \
              --max-sort-bytes 1G \
              --split-margin 2.0

The filesystem holds about 1.2G inodes. I tried to speed-up sorting with the last three arguments.
-- 
=======================
Heinrich Billich
ETH Zürich
Informatikdienste
Tel.: +41 44 632 72 56
heinrich.billich at id.ethz.ch
========================



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5254 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210208/c16dbf10/attachment-0001.bin>