<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1256">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1002700998;
mso-list-template-ids:-766460176;}
@list l1
{mso-list-id:1612398980;
mso-list-template-ids:1164981902;}
@list l1:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-start-at:0;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level3
{mso-level-start-at:0;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level4
{mso-level-start-at:0;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level5
{mso-level-start-at:0;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level6
{mso-level-start-at:0;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level7
{mso-level-start-at:0;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level8
{mso-level-start-at:0;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level9
{mso-level-start-at:0;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">Hi Paul,<br>
<br>
We were able to recreate the following scenario on our end and it is working as designed. I have created 5 files of 20MB each and migrated it to the cloud and then recall them via policy. When you recall the files using policy it decodes the %2F to / character
as generated in the policy-list file and when you try to recall manually using the path generated by policy it will fail as it is not the actual path. Below is the output attached:<br>
<br>
1. [root@c350f3u1 ~]# mmapplypolicy /gpfs0 -f /gpfs0/s3default/anujnhm/policy-list -I defer -L 3 -P recallFromCloud.policy.template<br>
[I] Summary of Rule Applicability and File Choices:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"> 0 5 20480 5 102400 102400 RULE 'dmrecall1' MIGRATE FROM POOL 'mcstore' TO POOL 'system'
WHERE(.)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">[I] Filesystem objects with no applicable rules: 349.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">[I] GPFS Policy Decisions and File Choice Totals:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">Chose to migrate 102400KB: 5 of 5 candidates;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">102400KB of chosen data is illplaced or illreplicated;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">Predicted Data Pool Utilization in KB and %:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">Pool_Name KB_Occupied KB_Total Percent_Occupied<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">system 28884992 31457280 91.822916667%<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">[I] Because some data is illplaced or illreplicated, predicted pool utilization may be negative and/or misleading!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">[I] 2022-07-28@05:00:07.154 Policy execution. 0 files dispatched.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">[I] A total of 0 files have been migrated, deleted or processed by an EXTERNAL EXEC/script;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"> 0 'skipped' files and/or errors.<br>
<br>
<br>
2. [root@c350f3u1 anujnhm]# head policy-list.recall.mcstore <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">156416 97874145 0 -- %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">156417 1726931404 0 -- %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">156418 1426209538 0 -- %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">156419 1880128499 0 -- %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">156420 1841735731 0 -- %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_5<br>
<br>
<br>
3. [root@c350f3u1 anujnhm]# mmcloudgateway files recall %2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">mmcloudgateway: Internal cloud services returned an error:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">file system object /gpfs0/s3default/anujnhm/%2Fgpfs0%2Fs3default%2Fanujnhm%2F20MB_file_2 does not exist<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">mmcloudgateway: Command failed. Examine previous error messages to determine cause.<br>
<br>
<br>
When TCT recalls files directly via policy, we do %2F decoding that the policy adds, but when we do explicit recalls using the standalone command, we do not do any decoding, because we don't expect files to be passed with any decoding to the explicit command.<br>
<br>
Thanks,<br>
Anuj Agnihotri<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-family:"Calibri",sans-serif;color:black">From:
</span></b><span style="font-family:"Calibri",sans-serif;color:black">Amey P Gokhale <amey.gokhale@in.ibm.com><br>
<b>Date: </b>Friday, 22 July 2022 at 2:42 PM<br>
<b>To: </b>Anuj Agnihotri <Anuj.Agnihotri@ibm.com><br>
<b>Subject: </b>FW: [EXTERNAL] Re: [gpfsug-discuss] mass recall from on-prem COS using a policy<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US">FYI -</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-family:"Arial",sans-serif;color:#7C7C5F">AMEY GOKHALE</span></b><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Senior Software Engineer Spectrum Scale</span></b><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#336699">Phone:
</span></b><span style="font-size:8.0pt;font-family:"Arial",sans-serif">91-988 100 8675</span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#336699">E-mail:
</span></b><span style="font-size:8.0pt;font-family:"Arial",sans-serif"><a href="mailto:amey.gokhale@in.ibm.com" target="_blank"><span style="color:#555555">amey.gokhale@in.ibm.com</span></a></span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Huzefa H Pancha <hupancha@in.ibm.com>
<b>On Behalf Of </b>scale@us.ibm.com<br>
<b>Sent:</b> 22 July 2022 00:49<br>
<b>To:</b> gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
<b>Cc:</b> gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org>; Amey P Gokhale <amey.gokhale@in.ibm.com><br>
<b>Subject:</b> Re: [EXTERNAL] Re: [gpfsug-discuss] mass recall from on-prem COS using a policy</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Adding back Amey in the chain.</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Paul please explicitly add Amey in your replies.</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Regards, The Spectrum Scale (GPFS) team<br>
<br>
------------------------------------------------------------------------------------------------------------------<br>
If you feel that your question can benefit other users of Spectrum Scale (GPFS), then please post it to the public IBM developerWroks Forum at
<a href="https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479">https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479</a>.
<br>
<br>
If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
<br>
<br>
The forum is informally monitored as time permits and should not be used for priority messages to the Spectrum Scale (GPFS) team.</span><br>
<br>
<img border="0" width="16" height="16" style="width:.1666in;height:.1666in" id="Picture_x0020_1" src="cid:0__=0ABB0215DFFA788F8f9e8a93df938690918c0ABB0215DFFA788F@IBMMail.com" alt="Inactive hide details for "Paul Ward" ---20-07-2022 08.20.55 PM---I used the template policy, substituting where applicable: # "><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#424282">"Paul
Ward" ---20-07-2022 08.20.55 PM---I used the template policy, substituting where applicable: # more /gpfs/nhmfsa/custom/valentine-migr</span><br>
<br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">From: </span>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Paul Ward" <<a href="mailto:p.ward@nhm.ac.uk">p.ward@nhm.ac.uk</a>></span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">To: </span>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif">"gpfsug main discussion list" <<a href="mailto:gpfsug-discuss@gpfsug.org">gpfsug-discuss@gpfsug.org</a>></span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date: </span>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif">20-07-2022 08.20 PM</span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Subject:
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">[EXTERNAL] Re: [gpfsug-discuss] mass recall from on-prem COS using a policy</span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Sent by:
</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"gpfsug-discuss" <<a href="mailto:gpfsug-discuss-bounces@gpfsug.org">gpfsug-discuss-bounces@gpfsug.org</a>></span><o:p></o:p></p>
<div>
<div class="MsoNormal">
<hr size="0" width="100%" noshade="" style="color:#8091A5" align="left">
</div>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<span style="font-size:7.5pt;color:white">I used the template policy, substituting where applicable: # more /gpfs/nhmfsa/custom/valentine-migrations/policy/iac-workspace_axio-imager_2021.pol /*******************************************************************************
</span><br>
<span style="font-size:7.5pt;color:white">ZjQcmQRYFpfptBannerStart</span><br>
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">This Message Is From an External Sender
</span></b><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif">This message came from outside your organization.
</span><br>
<br>
<span style="font-size:7.5pt;color:white">ZjQcmQRYFpfptBannerEnd</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">I used the template policy, substituting where applicable:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"># more /gpfs/nhmfsa/custom/valentine-migrations/policy/iac-workspace_axio-imager_2021.pol</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">/*******************************************************************************</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* Licensed Materials - Property of IBM</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">*</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* OCO Source Materials</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">*</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* (C) Copyright IBM Corp. 2016-2017 All Rights Reserved</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">*</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* The source code for this program is not published or other-</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* wise divested of its trade secrets, irrespective of what has</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">* been deposited with the U.S. Copyright Office.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">*******************************************************************************/</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">define(</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> exclude_list,</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> (</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> FALSE</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> OR PATH_NAME LIKE '%/.mcstore/%'</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> )</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">RULE EXTERNAL POOL
<b>'migrate'</b> EXEC '/usr/lpp/mmfs/bin/mmcloudgateway files' OPTS '-F' ESCAPE '% -'</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">/*</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">RULE EXTERNAL POOL 'mcstore' EXEC '/usr/lpp/mmfs/bin/mmcloudgateway files' OPTS '-F' ESCAPE '% -'</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> RULE 'dmrecall1' MIGRATE FROM POOL 'mcstore' TO POOL 'system' WHERE</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> (</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> (</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> PATH_NAME LIKE '/gpfs/test_dir/%'</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> )</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> AND NOT (exclude_list)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> )</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">*/</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">RULE 'migrate_bulk_workspace_axioimager_2021' MIGRATE FROM POOL 'migrate' TO POOL 'data' WHERE</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">(</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> (</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> PATH_NAME LIKE '<b>/gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/%</b>'</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> )</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> AND NOT (exclude_list)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">-----------------------------------<br>
I changed to the folder listed in the path and ran this command, with the defer option:<br>
<br>
# pwd</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">/gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">]# mmapplypolicy $(pwd) -f $(pwd)/policy-lists -I
<b>defer</b> -L 3 -P /gpfs/nhmfsa/custom/valentine-migrations/policy/iac-workspace_axio-imager_2021.pol</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"><br>
The result of the policy was:<br>
<br>
[I] Summary of Rule Applicability and File Choices:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> 0 994 0 994 1566373796 1566373796 RULE 'migrate_bulk_workspace_axioimager_2021' MIGRATE FROM POOL 'migrate' TO POOL 'data' WHERE(.)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">[I] Filesystem objects with no applicable rules: 53.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">[I] GPFS Policy Decisions and File Choice Totals:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Chose to migrate 1566373796KB: 994 of 994 candidates;</span><br>
<b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">1566373796KB of chosen data is illplaced or illreplicated</span></b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">;</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Predicted Data Pool Utilization in KB and %:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Pool_Name KB_Occupied KB_Total Percent_Occupied</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">data 175459129252 245111980032 71.583253185%</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">system 0 0 0.000000000% (no user data)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">[I] Because some data is illplaced or illreplicated, predicted pool utilization may be negative and/or misleading!</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">[I] </span><a href="mailto:2022-07-20@14:22:23.943"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">2022-07-20@14:22:23.943</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> Policy
execution. 0 files dispatched.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">[I] A total of 0 files have been migrated, deleted or processed by an EXTERNAL EXEC/script;</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> 0 'skipped' files and/or errors.<br>
</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">---------------------------------------------------------------------<br>
Its selected about the right number, as there are :</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"># find -type f | wc -l</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">996<br>
</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">--------------------------------------------------------<br>
the head of the deferred output file list:<br>
15:27:41 [root@scale-sk-pn-1 2021]# head policy-lists.recall.migrate</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">9085980 2146238895 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FRobin Armstrong 2021%2FThumbs.db</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157518 903677244 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-113_RL_s0c0x0-8849y0-8889.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157519 321389117 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-113_TL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157520 1970925505 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-113_TL_s0c0x0-8846y0-8892.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157521 1773348368 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-113_TL_s0c0x0-8849y0-8889.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157522 2126423502 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_RL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157523 1701851598 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_RL_s0c0x0-6359y0-6656.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157524 1844809347 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_TL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157525 912638442 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_TL_s0c0x0-6359y0-6656.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157526 1280698844 0 -- %2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_TL_s0c0x0-6359y0-7609.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">
.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"><br>
--------------------------</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">If I try to recall the file from these paths:<br>
<br>
# mmcloudgateway files recall "%2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_TL_s0c0x0-6359y0-7609.tif"</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">mmcloudgateway: Internal cloud services returned an error:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">file system object
<b>/gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/%</b>2Fgpfs%2Fnhmfsa%2Fbulk%2Fshare%2Fworkspace%2Fiac-workspace%2Flight_microscopy%2Fzeiss_axioimager_m2%2F2021%2FMatt Loader 2021%2FML210215_OT-29_TL_s0c0x0-6359y0-7609.tif
does not exist</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">mmcloudgateway: Command failed. Examine previous error messages to determine cause.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">It seems the highlighted part is the current directory, as I tried from another location and it picked up the pwd I was in.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">-----------------------</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">If I Give it the correctly formed path, it works:<br>
# mmcloudgateway files recall "/gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_TL_s0c0x0-6359y0-7609.tif"</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">mmcloudgateway: Command completed.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">------------------------------------------</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">I also ran a policy just to identify migrated files:</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">define(ismigrated,</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> ( MISC_ATTRIBUTES LIKE '%V%') )</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Rule 'migrated' LIST 'migrated_list' WHERE ismigrated</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Rule 'resident' LIST 'resident_list' WHERE not (ismigrated)</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">----------------------------------------------------------------</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">The head of that file correctly forms the posix path
</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">15:35:38 [root@scale-sk-pn-1 2021]# head list.migrated_list</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">9085980 2146238895 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Robin Armstrong 2021/Thumbs.db</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157518 903677244 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-113_RL_s0c0x0-8849y0-8889.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157519 321389117 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-113_TL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157520 1970925505 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-113_TL_s0c0x0-8846y0-8892.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157521 1773348368 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-113_TL_s0c0x0-8849y0-8889.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157522 2126423502 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_RL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157523 1701851598 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_RL_s0c0x0-6359y0-6656.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157524 1844809347 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_TL.czi</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157525 912638442 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_TL_s0c0x0-6359y0-6656.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">30157526 1280698844 0 -- /gpfs/nhmfsa/bulk/share/workspace/iac-workspace/light_microscopy/zeiss_axioimager_m2/2021/Matt Loader 2021/ML210215_OT-29_TL_s0c0x0-6359y0-7609.tif</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Whats going on?</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">What I have done in the past is take the output from my ismigrated policy.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Put recall commands before each path.</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Divide the file into 4 parts, and run each part of one of our 4 protocol nodes.<br>
Manually doing what mmapplypolicy should do!</span><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> gpfsug-discuss <<a href="mailto:gpfsug-discuss-bounces@gpfsug.org">gpfsug-discuss-bounces@gpfsug.org</a>>
<b>On Behalf Of </b>Amey P Gokhale<b><br>
Sent:</b> 20 July 2022 07:55<b><br>
To:</b> <a href="mailto:scale@us.ibm.com">scale@us.ibm.com</a>; gpfsug main discussion list <<a href="mailto:gpfsug-discuss@gpfsug.org">gpfsug-discuss@gpfsug.org</a>><b><br>
Cc:</b> gpfsug-discuss <<a href="mailto:gpfsug-discuss-bounces@gpfsug.org">gpfsug-discuss-bounces@gpfsug.org</a>><b><br>
Subject:</b> Re: [gpfsug-discuss] mass recall from on-prem COS using a policy</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">Paul Recall policy looks to be correct. I see that it is in line with /opt/ibm/MCStore/samples/recallFromCloud.template.
</span><o:p></o:p></p>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo3">
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">At the end of policy run, what does the stats indicate, such as X Files migrated, Y skipped/error etc? I assume the path you have specified, has the files in non-resident state, which you
are trying to recall using the policy. Correct? Reason I ask this is because you mentioned around 650G data to be recalled, but policy seems to have chosen ~2.5G data across 4812 files. So if this path is actively used by users, it is likely that the files
may have been transparently recalled on access, and are in co-resident state already. Hence no data movement visible. </span>
<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo3">
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">In /var/MCStore/ras/logs/mcstore.log, do you see any recall specific errors, when you run the policy?</span>
<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo3">
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif">On our test setup, we are planning to run similar policy once, to see if error is reproducible. I will share what we find.</span><o:p></o:p></li></ol>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif"> </span><br>
<b><span style="font-family:"Arial",sans-serif;color:#7C7C5F">AMEY GOKHALE</span></b><br>
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Senior Software Engineer Spectrum Scale</span></b><br>
<b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#336699">Phone:
</span></b><span style="font-size:7.5pt;font-family:"Arial",sans-serif">91-988 100 8675</span><br>
<b><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#336699">E-mail:
</span></b><a href="mailto:amey.gokhale@in.ibm.com" target="_blank"><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#555555">amey.gokhale@in.ibm.com</span></a><br>
<span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> </span><br>
<b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> Huzefa H Pancha <</span><a href="mailto:hupancha@in.ibm.com"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">hupancha@in.ibm.com</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">>
<b>On Behalf Of </b></span><a href="mailto:scale@us.ibm.com"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">scale@us.ibm.com</span></a><b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"><br>
Sent:</span></b><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"> 19 July 2022 23:42<b><br>
To:</b> gpfsug main discussion list <</span><a href="mailto:gpfsug-discuss@gpfsug.org"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">gpfsug-discuss@gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">>; Amey
P Gokhale <</span><a href="mailto:amey.gokhale@in.ibm.com"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">amey.gokhale@in.ibm.com</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">><b><br>
Cc:</b> gpfsug-discuss <</span><a href="mailto:gpfsug-discuss-bounces@gpfsug.org"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">gpfsug-discuss-bounces@gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">><b><br>
Subject:</b> Re: [EXTERNAL] Re: [gpfsug-discuss] mass recall from on-prem COS using a policy</span><br>
<o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hi Amey,</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
Can you provide them guidance from TCT angle.</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
Regards, The Spectrum Scale (GPFS) team<br>
<br>
------------------------------------------------------------------------------------------------------------------<br>
If you feel that your question can benefit other users of Spectrum Scale (GPFS), then please post it to the public IBM developerWroks Forum at
</span><a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ibm.com%2Fdeveloperworks%2Fcommunity%2Fforums%2Fhtml%2Fforum%3Fid%3D11111111-0000-0000-0000-000000000479&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Ca62196787b154603fba908da6a1d564b%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637938971501969639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=NHOeiZDnIn96yNcqo%2FWXp5fT5Bt%2BLfMMfHWvTd1bSSM%3D&reserved=0"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479</span></a><span style="font-size:10.0pt;font-family:"Arial",sans-serif">.
<br>
<br>
If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
<br>
<br>
The forum is informally monitored as time permits and should not be used for priority messages to the Spectrum Scale (GPFS) team.</span><br>
<br>
<img border="0" width="16" height="16" style="width:.1666in;height:.1666in" id="Picture_x0020_3" src="cid:0__=0ABB0215DFFA788F8f9e8a93df938690918c0ABB0215DFFA788F@IBMMail.com" alt="Inactive hide details for "Paul Ward" ---19-07-2022 09.34.33 PM---Thank you. Ill-placed on ESS or COS?"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#424282">"Paul
Ward" ---19-07-2022 09.34.33 PM---Thank you. Ill-placed on ESS or COS?</span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F"><br>
From: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Paul Ward" <</span><a href="mailto:p.ward@nhm.ac.uk"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">p.ward@nhm.ac.uk</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">><span style="color:#5F5F5F"><br>
To: </span>"gpfsug main discussion list" <</span><a href="mailto:gpfsug-discuss@gpfsug.org"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss@gpfsug.org</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">><span style="color:#5F5F5F"><br>
Date: </span>19-07-2022 09.34 PM<span style="color:#5F5F5F"><br>
Subject: </span>[EXTERNAL] Re: [gpfsug-discuss] mass recall from on-prem COS using a policy<span style="color:#5F5F5F"><br>
Sent by: </span>"gpfsug-discuss" <</span><a href="mailto:gpfsug-discuss-bounces@gpfsug.org"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss-bounces@gpfsug.org</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">></span><o:p></o:p></p>
<div>
<div class="MsoNormal">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="left">
</div>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<span style="font-size:7.5pt;color:white"><br>
Thank you. Ill-placed on ESS or COS? I understood restriping was for NSDs, so that would be on our ESS not COS? The direction I want to move the files is from COS to ESS. We do not have AFM enabled, we are using TCT. <br>
ZjQcmQRYFpfptBannerStart</span><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
This Message Is From an External Sender </span></b><span style="font-size:7.5pt;font-family:"Arial",sans-serif"><br>
This message came from outside your organization. </span><br>
<span style="font-size:7.5pt;color:white"><br>
ZjQcmQRYFpfptBannerEnd</span><span style="font-size:10.0pt;font-family:"Calibri",sans-serif"><br>
Thank you.<br>
<br>
Ill-placed on ESS or COS?<br>
<br>
I understood restriping was for NSDs, so that would be on our ESS not COS?<br>
The direction I want to move the files is from COS to ESS.<br>
<br>
We do not have AFM enabled, we are using TCT.<br>
<b><br>
From:</b> gpfsug-discuss <</span><a href="mailto:gpfsug-discuss-bounces@gpfsug.org"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">gpfsug-discuss-bounces@gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">>
<b>On Behalf Of </b>IBM Spectrum Scale<b><br>
Sent:</b> 18 July 2022 20:35<b><br>
To:</b> gpfsug main discussion list <</span><a href="mailto:gpfsug-discuss@gpfsug.org"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">gpfsug-discuss@gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">>; Venkateswara
R Puvvada <</span><a href="mailto:vpuvvada@in.ibm.com"><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">vpuvvada@in.ibm.com</span></a><span style="font-size:10.0pt;font-family:"Calibri",sans-serif">><b><br>
Subject:</b> Re: [gpfsug-discuss] mass recall from on-prem COS using a policy</span><br>
<o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> "KB_Ill" shows how much data are ill placed or ill replicated. They can be resolved by mmrestripefs or mmrestripefile.<span style="color:#2F2F2F"><br>
Copying to AFM team regarding recall in AFM-COS environment</span></span><span style="font-family:"Courier New";color:#2F2F2F">.</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
<br>
Regards, The Spectrum Scale (GPFS) team<br>
<br>
------------------------------------------------------------------------------------------------------------------<br>
If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
</span><br>
<br>
<img border="0" width="16" height="16" style="width:.1666in;height:.1666in" id="Picture_x0020_5" src="cid:0__=0ABB0215DFFA788F8f9e8a93df938690918c0ABB0215DFFA788F@IBMMail.com" alt="Inactive hide details for "Paul Ward" ---07/12/2022 11:40:52 AM---Hi all, I need to recall from on-prem COS a folder with subfo"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#424282">"Paul
Ward" ---07/12/2022 11:40:52 AM---Hi all, I need to recall from on-prem COS a folder with subfolders and files, approximately 4600 fil</span><span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F"><br>
<br>
From: </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Paul Ward" <</span><a href="mailto:p.ward@nhm.ac.uk"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">p.ward@nhm.ac.uk</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">><span style="color:#5F5F5F"><br>
To: </span>"</span><a href="mailto:gpfsug-discuss@gpfsug.org"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss@gpfsug.org</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">" <</span><a href="mailto:gpfsug-discuss@gpfsug.org"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss@gpfsug.org</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">><span style="color:#5F5F5F"><br>
Date: </span>07/12/2022 11:40 AM<span style="color:#5F5F5F"><br>
Subject: </span>[EXTERNAL] [gpfsug-discuss] mass recall from on-prem COS using a policy<span style="color:#5F5F5F"><br>
Sent by: </span>"gpfsug-discuss" <</span><a href="mailto:gpfsug-discuss-bounces@gpfsug.org"><span style="font-size:7.5pt;font-family:"Arial",sans-serif">gpfsug-discuss-bounces@gpfsug.org</span></a><span style="font-size:7.5pt;font-family:"Arial",sans-serif">></span><o:p></o:p></p>
<div>
<div class="MsoNormal">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="left">
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
<br>
<span style="font-size:7.5pt;color:white"><br>
<br>
Hi all, I need to recall from on-prem COS a folder with subfolders and files, approximately 4600 files making up 656G. We have a policy that runs every 30 mins, and I added this line to it: <br>
ZjQcmQRYFpfptBannerStart</span><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
This Message Is From an External Sender </span></b><span style="font-size:7.5pt;font-family:"Arial",sans-serif"><br>
This message came from outside your organization. </span><span style="font-size:7.5pt;color:white"><br>
<br>
ZjQcmQRYFpfptBannerEnd</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
Hi all,<br>
<br>
I need to recall from on-prem COS a folder with subfolders and files, approximately 4600 files making up 656G.<br>
We have a policy that runs every 30 mins, and I added this line to it:<br>
<br>
RULE 'migrate_iac-workspace_Chem-labs' MIGRATE FROM POOl 'migrate' TO POOL 'data' WHERE PATH_NAME LIKE '/gpfs/nhmfsa/bulk/share/workspace/iac-workspace/Chem_Labs/%'<br>
<br>
Migrate is an external pool<br>
RULE EXTERNAL POOL 'migrate' EXEC '/usr/lpp/mmfs/bin/mmcloudgateway files' OPTS '-F' ESCAPE '% -'<br>
<br>
And data is the default placement pool<br>
RULE 'Placement' SET POOL 'data'<br>
<br>
When it runs it identifies matching files:<br>
Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule<br>
13 4846 2491152 4846 2491152 7056 RULE 'migrate_iac-workspace_Chem-labs' MIGRATE FROM POOL 'migrate' TO POOL 'data' WHERE(.)<br>
<br>
I cant find what KB_ Ill means but nothing is migrating.<br>
<br>
Migrations in other policies work, but this is a recall policy.<br>
>From this document </span><a href="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ibm.com%2Fdocs%2Fen%2Fspectrum-scale%2F5.1.3%3Ftopic%3Dpools-migrate-recall-external&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Ca62196787b154603fba908da6a1d564b%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637938971501969639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=lekQJv2rgi%2B8CKoheGryeF0HaSmZIxKaTEBofyAMypY%3D&reserved=0"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#0563C1">https://www.ibm.com/docs/en/spectrum-scale/5.1.3?topic=pools-migrate-recall-external</span></a><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
It suggests it should be invoked as a recall.<br>
<br>
Any ideas?<br>
<br>
Kindest regards,<br>
Paul<br>
<br>
Paul Ward<br>
TS Infrastructure Architect<br>
Natural History Museum<br>
T: 02079426450<br>
E: </span><a href="mailto:p.ward@nhm.ac.uk"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#0563C1">p.ward@nhm.ac.uk</span></a><br>
<img border="0" width="197" height="107" style="width:2.052in;height:1.1145in" id="Picture_x0020_7" src="cid:1__=0ABB0215DFFA788F8f9e8a93df938690918c0ABB0215DFFA788F@IBMMail.com" alt="A picture containing drawing
Description automatically generated"><span style="font-size:10.0pt;font-family:"Courier New""><br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org</span><u><span style="color:blue"><br>
</span></u><a href="https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss_gpfsug.org&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Ca62196787b154603fba908da6a1d564b%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637938971501969639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=kYuZCRoUCPgMVc5zNJiLI%2BDAO3wh8UGx6O3%2F9iBDkeM%3D&reserved=0"><span style="font-size:10.0pt;font-family:"Courier New"">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Courier New""> </span><br>
<br>
<span style="font-size:10.0pt;font-family:"Courier New""><br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<u><span style="color:blue"><br>
</span></u></span><a href="https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss_gpfsug.org&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Ca62196787b154603fba908da6a1d564b%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637938971501969639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=kYuZCRoUCPgMVc5zNJiLI%2BDAO3wh8UGx6O3%2F9iBDkeM%3D&reserved=0"><span style="font-size:10.0pt;font-family:"Courier New"">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</span></a><span style="font-size:10.0pt;font-family:"Courier New""> <tt>_______________________________________________</tt><br>
<tt>gpfsug-discuss mailing list</tt><br>
<tt>gpfsug-discuss at gpfsug.org</tt><br>
<tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a> </tt><br>
<br>
<br>
</span><o:p></o:p></p>
</div>
</body>
</html>