<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Replying to a 3 year old message I sent, hoping that in the last couple of years that Scale has added some ILM extensions into the policy engine that I have missed, or somehow didn't notice?</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Just ran into a file with an 'unbalanced' flag and I REALLY don't want to have to mmlsattr everything. AGAIN. /facepalm </div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: "Times New Roman", Times, serif; font-size: 12pt;">IBM? Bueller? Bueller? </span><br>
</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
When everyone answers: "No", I'm guessing this needs to be a request for improvement/enhancement? </div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Ed Wahl</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Ohio Supercomputer Center</div>
<div style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Edward Wahl <ewahl@osc.edu><br>
<b>Sent:</b> Friday, February 2, 2018 3:23 PM<br>
<b>To:</b> John Hearns <john.hearns@asml.com><br>
<b>Cc:</b> gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] policy ilm features?</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText"><br>
Thanks John, this was the path I was HOPING to go down as I do similar things<br>
already, but there appears to be no extended attribute in ILM for what I want.<br>
Data block replication flag exists in the ILM, but not MetaData, or balance.<br>
<br>
Yet these states ARE reported by mmlsattr, so there must be a flag somewhere.<br>
<br>
<br>
bad MD replication & balance example:<br>
<br>
mmlsattr -L /fs/scratch/sysp/ed/180days.pol <br>
file name: /fs/scratch/sysp/ed/180days.pol<br>
metadata replication: 1 max 2<br>
data replication: 1 max 2<br>
flags: illreplicated,unbalanced<br>
Encrypted: yes<br>
<br>
File next to it for comparison. note proper MD replication and balance.<br>
<br>
mmlsattr -L /fs/scratch/sysp/ed/120days.pol <br>
file name: /fs/scratch/sysp/ed/120days.pol<br>
metadata replication: 2 max 2<br>
data replication: 1 max 2<br>
flags: <br>
Encrypted: yes<br>
<br>
misc_attributes flags from a policy run showing no difference in status:<br>
FJAEu -- /fs/scratch/sysp/ed/180days.pol<br>
FJAEu -- /fs/scratch/sysp/ed/120days.pol<br>
<br>
<br>
File system has MD replication enabled, but not Data, so ALL files show "J" ilm<br>
flag<br>
<br>
mmlsfs scratch -m<br>
flag value description<br>
------------------- ------------------------ -----------------------------------<br>
-m 2 Default number of metadata replicas<br>
mmlsfs scratch -r<br>
flag value description<br>
------------------- ------------------------ -----------------------------------<br>
-r 1 Default number of data replicas<br>
<br>
<br>
I poked around a little trying to find out if perhaps using GetXattr would<br>
work and show me what I wanted, it does not. All I sem to be able to get is the<br>
File Encryption Key.<br>
<br>
<br>
I was hoping perhaps someone had found a cheaper way for this to work rather<br>
than hundreds of millions of 'mmlsattr' execs. :-(<br>
<br>
On the plus side, I've only run across a few of these and all appear to be<br>
from before we did the MD replication and re-striping. On the minus, I have NO<br>
idea where they are, and they appears to be on both of our filesystems. So<br>
several hundred million files to check. <br>
<br>
Ed<br>
<br>
<br>
On Mon, 22 Jan 2018 08:29:42 +0000<br>
John Hearns <john.hearns@asml.com> wrote:<br>
<br>
> Ed,<br>
> This is not a perfect answer. You need to look at policies for this. I have<br>
> been doing something similar recently.<br>
> <br>
> Something like:<br>
> <br>
> RULE 'list_file' EXTERNAL LIST 'all-files' EXEC<br>
> '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files'<br>
> SHOW( varchar(kb_allocated) || ' ' || varchar(file_size) || ' ' ||<br>
> varchar(misc_attributes) || ' ' || name || ' ' || fileset_name ) WHERE<br>
> REGEX(misc_attributes,'[J]')<br>
> <br>
> <br>
> So this policy shows the kbytes allocates, file size, the miscellaneous<br>
> attributes, name and fileset name For all files with miscellaneous<br>
> attributes of 'J' which means 'Some data blocks might be ill replicated'<br>
> <br>
> <br>
> <br>
> <br>
> -----Original Message-----<br>
> From: gpfsug-discuss-bounces@spectrumscale.org<br>
> [<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>] On Behalf Of Edward Wahl<br>
> Sent: Friday, January 19, 2018 10:38 PM To: gpfsug-discuss@spectrumscale.org<br>
> Subject: [gpfsug-discuss] policy ilm features?<br>
> <br>
> <br>
> This one has been on my list a long time so I figured I'd ask here first<br>
> before I open an apar or request an enhancement (most likely).<br>
> <br>
> Is there a way using the policy engine to determine the following?<br>
> <br>
> -metadata replication total/current<br>
> -unbalanced file<br>
> <br>
> Looking to catch things like this that stand out on my filesystem without<br>
> having to run several hundred million 'mmlsattr's.<br>
> <br>
> metadata replication: 1 max 2<br>
> flags: unbalanced<br>
> <br>
> Ed<br>
> <br>
> <br>
> <br>
> --<br>
> <br>
> Ed Wahl<br>
> Ohio Supercomputer Center<br>
> 614-292-9302<br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at spectrumscale.org<br>
> <a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0">
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0</a><br>
> -- The information contained in this communication and any attachments is<br>
> confidential and may be privileged, and is for the sole use of the intended<br>
> recipient(s). Any unauthorized review, use, disclosure or distribution is<br>
> prohibited. Unless explicitly stated otherwise in the body of this<br>
> communication or the attachment thereto (if any), the information is provided<br>
> on an AS-IS basis without any express or implied warranties or liabilities.<br>
> To the extent you are relying on this information, you are doing so at your<br>
> own risk. If you are not the intended recipient, please notify the sender<br>
> immediately by replying to this message and destroy all copies of this<br>
> message and any attachments. Neither the sender nor the company/group of<br>
> companies he or she represents shall be liable for the proper and complete<br>
> transmission of the information contained in this communication, or for any<br>
> delay in its receipt. _______________________________________________<br>
> gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
-- <br>
<br>
Ed Wahl<br>
Ohio Supercomputer Center<br>
614-292-9302<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</div>
</span></font></div>
</body>
</html>