[gpfsug-discuss] policy ilm features?

Tue Feb 2 21:26:42 GMT 2021

My issues were never specifically the Unbalanced flag, originally I ran into ilm/policy issues with our metadata and illreplicated files.  We were working to expand our Metadata storage at the time uner 4.2.3.xx and had added a number of SSDs to the array.  But AFTER a restripe files were still illreplicated.  I then discovered that the policy engine had no way to tell me what files were and were not replicated.   Just hoping that more ILM info had been added to the policy engine since I ran into this, and seeing the unbalanced files jogged my memory to look at the docs again, where I didn't see anything in 5.x and ask the list as well.   This isn't a big deal, more curiousity on my part.

I see you have some of the original thread attached, so perhaps take a glance and see if it makes sense?  I

Now if you REALLY want to step into a mine field, go find my thread on SKLM usage and GPFS/SS.   Every single respondent to my question about filling up their SKLM logs with errors was a positive.  And SKLM L2 and L3 support swears GPFS/SS is using SKLM wrong...

Ed Wahl
OSC

________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Frederick Stock <stockf at us.ibm.com>
Sent: Tuesday, February 2, 2021 1:09 PM
To: gpfsug-discuss at spectrumscale.org <gpfsug-discuss at spectrumscale.org>
Cc: gpfsug-discuss at spectrumscale.org <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] policy ilm features?

Hello Ed.  Jordan contacted me about the question you are posing so I am responding to your message.  Could you please provide clarification as to why the existence of the unbalanced flag is of a concern, or why you would want to know all the files that have this flag set?  The flag would be cleared once the file was rebalanced either through normal access or through the execution of the mmrestripefs/mmrestripefile commands.

Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
stockf at us.ibm.com

----- Original message -----
From: "Wahl, Edward" <ewahl at osc.edu>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] policy ilm features?
Date: Tue, Feb 2, 2021 11:52 AM

Replying to a 3 year old message I sent, hoping that in the last couple of years that Scale has added some ILM extensions into the policy engine that I have missed, or somehow didn't notice?
Just ran into a file with an 'unbalanced' flag and I REALLY don't want to have to mmlsattr everything. AGAIN. /facepalm

IBM?  Bueller? Bueller?

When everyone answers: "No", I'm guessing this needs to be a request for improvement/enhancement?

Ed Wahl
Ohio Supercomputer Center

________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Edward Wahl <ewahl at osc.edu>
Sent: Friday, February 2, 2018 3:23 PM
To: John Hearns <john.hearns at asml.com>
Cc: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] policy ilm features?

Thanks John, this was the path I was HOPING to go down as I do similar things
already, but there appears to be no extended attribute in ILM for what I want.
Data block replication flag exists in the ILM, but not MetaData, or balance.

Yet these states ARE reported by mmlsattr, so there must be a flag somewhere.

bad MD replication & balance example:

 mmlsattr -L /fs/scratch/sysp/ed/180days.pol
file name:            /fs/scratch/sysp/ed/180days.pol
metadata replication: 1 max 2
data replication:     1 max 2
flags:                illreplicated,unbalanced
Encrypted:            yes

File next to it for comparison. note proper MD replication and balance.

 mmlsattr -L  /fs/scratch/sysp/ed/120days.pol
file name:            /fs/scratch/sysp/ed/120days.pol
metadata replication: 2 max 2
data replication:     1 max 2
flags:
Encrypted:            yes

misc_attributes flags from a policy run showing no difference in status:
FJAEu -- /fs/scratch/sysp/ed/180days.pol
FJAEu -- /fs/scratch/sysp/ed/120days.pol

File system has MD replication enabled, but not Data, so ALL files show "J" ilm
flag

mmlsfs scratch -m
flag                value                    description
------------------- ------------------------ -----------------------------------
 -m                 2                        Default number of metadata replicas
mmlsfs scratch -r
flag                value                    description
------------------- ------------------------ -----------------------------------
 -r                 1                        Default number of data replicas

I poked around a little trying to find out if perhaps using GetXattr would
work and show me what I wanted, it does not. All I sem to be able to get is the
File Encryption Key.

I was hoping perhaps someone had found a cheaper way for this to work rather
than hundreds of millions of 'mmlsattr' execs.  :-(

On the plus side, I've only run across a few of these and all appear to be
from before we did the MD replication and re-striping.  On the minus, I have NO
idea where they are, and they appears to be on both of our filesystems.  So
several hundred million files to check.

Ed

On Mon, 22 Jan 2018 08:29:42 +0000
John Hearns <john.hearns at asml.com> wrote:

> Ed,
> This is not a perfect answer. You need to look at policies for this. I have
> been doing something similar recently.
>
> Something like:
>
> RULE 'list_file' EXTERNAL LIST 'all-files' EXEC
> '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files'
> SHOW( varchar(kb_allocated) || '  ' || varchar(file_size) || ' ' ||
> varchar(misc_attributes) || ' ' || name || ' ' || fileset_name  ) WHERE
> REGEX(misc_attributes,'[J]')
>
>
> So this policy shows the kbytes allocates, file size, the miscellaneous
> attributes, name and fileset name For all files with  miscellaneous
> attributes of 'J'   which means 'Some data blocks might be ill replicated'
>
>
>
>
> -----Original Message-----
> From: gpfsug-discuss-bounces at spectrumscale.org
> [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Edward Wahl
> Sent: Friday, January 19, 2018 10:38 PM To: gpfsug-discuss at spectrumscale.org
> Subject: [gpfsug-discuss] policy ilm features?
>
>
> This one has been on my list a long time so I figured I'd ask here first
> before I open an apar or request an enhancement (most likely).
>
>  Is there a way using the policy engine to determine the following?
>
> -metadata replication total/current
> -unbalanced file
>
> Looking to catch things like this that stand out on my filesystem without
> having to run several hundred million 'mmlsattr's.
>
> metadata replication: 1 max 2
> flags:                unbalanced
>
> Ed
>
>
>
> --
>
> Ed Wahl
> Ohio Supercomputer Center
> 614-292-9302
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0<https://urldefense.com/v3/__https://emea01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fgpfsug.org*2Fmailman*2Flistinfo*2Fgpfsug-discuss&data=01*7C01*7Cjohn.hearns*40asml.com*7C056e34c5a8df4d8f10fd08d55f91e73c*7Caf73baa8f5944eb2a39d93e96cad61fc*7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM*2B8pNrZElImSZeZbit8*3D&reserved=0__;JSUlJSUlJSUlJSUlJSU!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foRAVfkHJ$>
> -- The information contained in this communication and any attachments is
> confidential and may be privileged, and is for the sole use of the intended
> recipient(s). Any unauthorized review, use, disclosure or distribution is
> prohibited. Unless explicitly stated otherwise in the body of this
> communication or the attachment thereto (if any), the information is provided
> on an AS-IS basis without any express or implied warranties or liabilities.
> To the extent you are relying on this information, you are doing so at your
> own risk. If you are not the intended recipient, please notify the sender
> immediately by replying to this message and destroy all copies of this
> message and any attachments. Neither the sender nor the company/group of
> companies he or she represents shall be liable for the proper and complete
> transmission of the information contained in this communication, or for any
> delay in its receipt. _______________________________________________
> gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$>

--

Ed Wahl
Ohio Supercomputer Center
614-292-9302
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210202/dea769d4/attachment-0002.htm>