[gpfsug-discuss] [EXTERNAL] Re: Handling bad file names in policies?

[Peter Childs]

We've had this same issue with characters that are fine in Scale but Protect can't handle. Normally its because some script has embedded a newline in the middle of a file name, and normally we end up renaming that file by inode number

find . -inum 9975226749 -exec mv {} badfilename \;

mostly because we can't even type the filename at the command prompt.

However its not always just new line characters currently we've got a few files with unprintable characters in it. but its normally less than 50 files every few months, so is easy to handle manually.

I normally end up looking at /data/mmbackup.unsupported which is the standard output from mmapplypolicy and extracting the file names from it and emailing the users concerned to assist them in working out what went wrong.

I guess you could automate the parsing of this file at the end of the backup process and do something interesting with it.


Peter Childs



________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Simon Thompson <s.j.thompson at bham.ac.uk>
Sent: Monday, October 11, 2021 9:35 AM
To: gpfsug main discussion list
Subject: [EXTERNAL] Re: [gpfsug-discuss] Handling bad file names in policies?

CAUTION: This email originated from outside of QMUL. Do not click links or open attachments unless you recognise the sender and know the content is safe.


We have both:

  WILDCARDSARELITERAL       yes
  QUOTESARELITERAL          yes

Set. And use --noquote for mmbackup, the backup runs, but creates a file:

/filesystem/mmbackup.unsupported.CLIENTNAME

Which contains a list of files that are not backed up due to \n in the filename.

So it doesn't break backup, but they don't get backed up either. I believe this is because the TSM client can't back the file up rather than mmbackup no longer allowing them. I had an RFE at some point to get dsmc changed ... but it got closed WONTFIX.

Simon

On 09/10/2021, 10:09, "gpfsug-discuss-bounces at spectrumscale.org on behalf of Jonathan Buzzard" <gpfsug-discuss-bounces at spectrumscale.org on behalf of jonathan.buzzard at strath.ac.uk> wrote:

    On 08/10/2021 19:14, Wahl, Edward wrote:

    > This goes back as far as I can recall to <=GPFS 3.5 days. And no, I
    > cannot recall what version of TSM-EE that was.   But newline has been
    > the only stopping point, for what seems like forever. Having filed
    > many an mmbackup bug, I don't recall ever crashing on filenames.
    > (tons of OTHER reasons, but not character set)   We even generate an
    > error report from this and email users to fix it. We accept basically
    > almost everything else, and I have to say, we see some really crazy
    > things sometimes.   I think my current favorite is the full windows
    > paths as a filename. (eg:
    > "Y:\Temp\temp\290\work\0\Material_ERTi-5.in" )
    >

    I will have to do a test but I am sure newlines have worked just fine in
    the past. At the very least they have not stopped an entire backup from
    working when using dsmc incr.

    Now mmbackup that's a different kettle of fish. If you have not seen
    mmbackup fail entirely because of a random "special" character you
    simply have not been using it long enough :-)

    For the longest of times I would simply not go anywhere near it because
    it was not fit for purpose.

    >
    > Current IBM documentation doesn't go backwards past 4.2 but it says:
    >
    > "For IBM Spectrum Scale™ file systems with special characters
    > frequently used in the names of files or directories, backup failures
    > might occur. Known special characters that require special handling
    > include: *, ?, ", ’, carriage return, and the new line character.
    >
    > In such cases, enable the Tivoli Storage Manager client options
    > WILDCARDSARELITERAL and QUOTESARELITERAL on all nodes that are used
    > in backup activities and make sure that the mmbackup option --noquote
    > is used when invoking mmbackup."
    >
    > So maybe we could handle newlines somehow.   But my lazy searches
    > didn't show what TSM doesn't accept.
    >

    We strongly advise our users (our GPFS file system is for an HPC system)
    in training not to use "special" characters. That is followed with a
    warning that if they do then we don't make any promises to backup their
    files :-)

     From time to time I run a dsmc incr in a screen and capture the output
    to a log file and then look at the list of failed files and prompt users
    to "fix" them. Though sometimes I just "fix" them myself if the
    correction is going to be obvious and then email them to tell them what
    has happened.


    JAB.

    --
    Jonathan A. Buzzard                         Tel: +44141-5483420
    HPC System Administrator, ARCHIE-WeSt.
    University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
    _______________________________________________
    gpfsug-discuss mailing list
    gpfsug-discuss at spectrumscale.org
    http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss