[gpfsug-discuss] Handling bad file names in policies?

[Wahl, Edward]

Sadly the ESCAPE only works for EXTERNAL LISTs, correct?   Not sure that I can easily modify an EXERNAL LIST to do what I want, which is a LIST policy using MISC_ATTRIBUTES and find all files without X, etc.
And using mmlsattr on hundreds of millions of files will take until the next millennium, so I really would like to stick with the policy engine.  Perhaps I can do some RULE 1 feeds RULE 2 type thing?

Sort of thing I’m looking at:

define( immut, MISC_ATTRIBUTES LIKE '%X%')
RULE 'listimmut' LIST 'not-immut' WHERE NOT (exclude_list) and NOT (immut)


Ed Wahl
OSC


From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of Olaf Weiser
Sent: Tuesday, October 5, 2021 2:10 AM
To: gpfsug-discuss at spectrumscale.org
Cc: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] Handling bad file names in policies?

Hi  Ed,

not a ready to run for "everything".. but just to remind, there is an ESCAPE statement
by this you can

 cat policy2
RULE EXTERNAL LIST 'allfiles' EXEC '/var/mmfs/etc/list.exe'  ESCAPE '%/#'

and turn a file name into smth , what a policy can use

I haven't used it for a while , but here is an example from a while ago .. ;-)

[root at c25m4n03 stupid_files]# ll
total 0
-rw-r--r-- 1 root root 21 Mar 22 03:44 dämlicher filename
-rw-r--r-- 1 root root  2 Mar 22 03:59 üöä???ßß spacefilen
[root at c25m4n03 stupid_files]#


policy:
101378 247907919 0   -- /gpfs/fpofs/files/stupid_files/d%C3%A4mlicher%20filename
101381 1945364096 0   -- /gpfs/fpofs/files/stupid_files/%C3%BC%C3%BC%C3%BC%C3%B6%C3%B6%C3%A4%C3%A4%3F%3F%3F%C3%9F%C3%9F%20spacefilename
[I]2013-03-22 at 13:12:58.687<mailto:2013-03-22 at 13:12:58.687> Policy execution. 2 files dispatched.

 verify with policy  (ESCAPE '%/ä ')
101378 247907919 0   -- /gpfs/fpofs/files/stupid_files/dämlicher filename
[...]


hope this helps..
cheers




----- Ursprüngliche Nachricht -----
Von: "Jonathan Buzzard" <jonathan.buzzard at strath.ac.uk<mailto:jonathan.buzzard at strath.ac.uk>>
Gesendet von: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
An: gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>
CC:
Betreff: [EXTERNAL] Re: [gpfsug-discuss] Handling bad file names in policies?
Datum: Di, 5. Okt 2021 01:29

On 04/10/2021 23:23, Wahl, Edward wrote:

> I know I've run into this before way back, but my notes on how I solved
> this aren't getting the job done in Scale 5.0.5.8 and my notes are from
> 3.5.  😉
> Anyone know a way to get a LIST policy to properly feed bad filenames
> into the output or an external script?
>
> When I say bad I mean things like control characters, spaces, etc.   Not
> concerned about the dreaded 'newline' as we force users to fix those or
> the files do not get backed up in Tivoli.
>

Since when? Last time I checked which was admittedly circa 2008, TSM
would backup files with newlines in them no problem. mmbackup on the
other hand in that time frame would simply die and backup nothing if
there was a single file on the file system with a newline in it.

I would take a look at the mmbackup scripts which can handle such stuff
(least ways in >4.2) which would also suggest dsmc can handle it.

As an aside I now think I know how you end up with newlines in file
names. Basically you cut and paste the file name complete with newlines
(most likely at the end) into a text field when saving the file.
Personally I think any program should baulk at that point but what do I
know.


JAB.

--
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://urldefense.com/v3/__http:/gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!gbBLWYl7S7BX4mw1st0Uqn0jAON438v_xU5im5y1VOf3admLYLebW4C0k2nP$>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20211008/6d7eabcc/attachment-0002.htm>