[gpfsug-discuss] Handling bad file names in policies?

[Alec]

Well....

How about:

define(DISPLAY_NEWLINE,[CASE WHEN ($1) *HAS NEWLINE* THEN *REPLACE NEWLINE
WITH ALTERNATE CHARACTER* ELSE varchar(1) END])

Define your show to have the DISPLAY_NEWLINE in place of the file name?


Sorry I don't know offhand how to do the find newline and replace newline
sql string code, I don't have gpfs at home sadly.

On Fri, Oct 8, 2021, 1:42 PM Wahl, Edward <ewahl at osc.edu> wrote:

> Sadly the ESCAPE only works for EXTERNAL LISTs, correct?   Not sure that
> I can easily modify an EXERNAL LIST to do what I want, which is a LIST
> policy using MISC_ATTRIBUTES and find all files without X, etc.
>
> And using mmlsattr on hundreds of millions of files will take until the
> next millennium, so I really would like to stick with the policy engine.  Perhaps
> I can do some RULE 1 feeds RULE 2 type thing?
>
>
>
> Sort of thing I’m looking at:
>
>
>
> define( immut, MISC_ATTRIBUTES LIKE '%X%')
>
> RULE 'listimmut' LIST 'not-immut' WHERE NOT (exclude_list) and NOT (immut)
>
>
>
>
>
> Ed Wahl
>
> OSC
>
>
>
>
>
> *From:* gpfsug-discuss-bounces at spectrumscale.org <
> gpfsug-discuss-bounces at spectrumscale.org> *On Behalf Of *Olaf Weiser
> *Sent:* Tuesday, October 5, 2021 2:10 AM
> *To:* gpfsug-discuss at spectrumscale.org
> *Cc:* gpfsug-discuss at spectrumscale.org
> *Subject:* Re: [gpfsug-discuss] Handling bad file names in policies?
>
>
>
> Hi  Ed,
>
>
>
> not a ready to run for "everything".. but just to remind, there is an
> ESCAPE statement
>
> by this you can
>
>
>
>  cat policy2
> RULE EXTERNAL LIST 'allfiles' EXEC '/var/mmfs/etc/list.exe'  ESCAPE '%/#'
>
>
>
> and turn a file name into smth , what a policy can use
>
>
>
> I haven't used it for a while , but here is an example from a while ago ..
> ;-)
>
>
>
> [root at c25m4n03 stupid_files]# ll
> total 0
> -rw-r--r-- 1 root root 21 Mar 22 03:44 dämlicher filename
> -rw-r--r-- 1 root root  2 Mar 22 03:59 üöä???ßß spacefilen
> [root at c25m4n03 stupid_files]#
>
>
>
>
>
> policy:
>
> 101378 247907919 0   --
> /gpfs/fpofs/files/stupid_files/d%C3%A4mlicher%20filename
> 101381 1945364096 0   --
> /gpfs/fpofs/files/stupid_files/%C3%BC%C3%BC%C3%BC%C3%B6%C3%B6%C3%A4%C3%A4%3F%3F%3F%C3%9F%C3%9F%20spacefilename
> [I]2013-03-22 at 13:12:58.687 Policy execution. 2 files dispatched.
>
>
>
>
>  verify with policy  (ESCAPE '%/ä ')
>
> 101378 247907919 0   -- /gpfs/fpofs/files/stupid_files/dämlicher filename
> [...]
>
>
>
>
>
> hope this helps..
>
> cheers
>
>
>
>
>
>
>
>
>
> ----- Ursprüngliche Nachricht -----
> Von: "Jonathan Buzzard" <jonathan.buzzard at strath.ac.uk>
> Gesendet von: gpfsug-discuss-bounces at spectrumscale.org
> An: gpfsug-discuss at spectrumscale.org
> CC:
> Betreff: [EXTERNAL] Re: [gpfsug-discuss] Handling bad file names in
> policies?
> Datum: Di, 5. Okt 2021 01:29
>
>
> On 04/10/2021 23:23, Wahl, Edward wrote:
>
> > I know I've run into this before way back, but my notes on how I solved
> > this aren't getting the job done in Scale 5.0.5.8 and my notes are from
> > 3.5.  😉
> > Anyone know a way to get a LIST policy to properly feed bad filenames
> > into the output or an external script?
> >
> > When I say bad I mean things like control characters, spaces, etc.   Not
> > concerned about the dreaded 'newline' as we force users to fix those or
> > the files do not get backed up in Tivoli.
> >
>
> Since when? Last time I checked which was admittedly circa 2008, TSM
> would backup files with newlines in them no problem. mmbackup on the
> other hand in that time frame would simply die and backup nothing if
> there was a single file on the file system with a newline in it.
>
> I would take a look at the mmbackup scripts which can handle such stuff
> (least ways in >4.2) which would also suggest dsmc can handle it.
>
> As an aside I now think I know how you end up with newlines in file
> names. Basically you cut and paste the file name complete with newlines
> (most likely at the end) into a text field when saving the file.
> Personally I think any program should baulk at that point but what do I
> know.
>
>
> JAB.
>
> --
> Jonathan A. Buzzard                         Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> <https://urldefense.com/v3/__http:/gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!gbBLWYl7S7BX4mw1st0Uqn0jAON438v_xU5im5y1VOf3admLYLebW4C0k2nP$>
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20211008/a5e83f80/attachment-0002.htm>