[gpfsug-discuss] mmbackup valiating the backups

[Stephan Graf]

Hi,

I also recommend to use the '-q' option in a regularly way to be sure 
not to get some kind of 'split brain' situation: The TSM DB content and 
the shadow database.

If You run mmbackup on a large file system where a lot of changes are 
ongoing, we always see in the final report of mmbackup that some files 
failed.
As far as I remember the situation was, that the file was deleted by the 
user before the dsmc was started.
But it is hard to check it for all files so we ignore those failures. 
But it could be that something different is the root cause of not 
backing up this file. In the past we saw it when users used strange 
characters in their file name.

For this it would be nice to get some kind of better report from mmbackup.
So for example a filelist of all files which failed. May be with the reason
- file not found
- dsmc backup failed
- ...

This is a question to the IBM Devs: What do You think? Is it worth of 
opening an IBM IDEA?

Stephan


On 1/16/26 07:53, Timm Stamer wrote:
> Hi ,
> 
> we're running mmbackup with -q option once a quarter to be sure
> everything is backed up.
> I think this is much simpler than your approach but maybe does not
> cover all your needs.
> 
> 
> -q
>      Performs a query operation before issuing mmbackup. The IBM Storage
> Protect server might have data stored already that is not recognized as
> having been backed up by mmbackup and its own shadow database. To
> properly compute the set of files that currently need to be backed up,
> mmbackup can perform an IBM Storage Protect query and process the
> results to update its shadow database. Use the -q switch to perform
> this query and then immediately commence the requested backup
> operation.
> 
> https://www.ibm.com/docs/en/storage-scale/5.2.3?topic=reference-mmbackup-command
> 
> 
> [...]
> we=$(LC_TIME=C date +%A)
> dm=$(date +%d)
> dmonth=$(date +%m)
> checkMonths=("01" "04" "07" "10")
> [[ ${checkMonths[@]} =~ $dmonth ]] && [ "$we" = "Friday" ] && [ "$dm" -le 7 ] && QUERYBACKUP="-q"
> 
> mmbackup ... ${QUERYBACKUP} ...
> [...]
> 
> 
> Kind regards
> 
> Timm Stamer
> 
> 
> 
> 
> Am Donnerstag, dem 15.01.2026 um 18:21 +0000 schrieb Peter Childs:
>>
>> Hi All,
>>
>> We use mmbackup to backup our Scale Storage to two IBM Protect
>> Instances.
>>
>> We would like to validate our backups to ensure that the files we
>> have really are backed up and we don't have any problems.
>>
>> So far I've worked out I can query the backups using "query backup --
>> detail" in Protect and get times and dates, From this I can compare
>> with `stat` and `ls` (or using mmapplypolicy (which should be
>> faster)) to check the three indexes match, and check the contents of
>> each directory. (Using --subdir=yes looks great on paper but the
>> output takes days to appear and checking it file by file can be done
>> incrementally and I can use mmapplypolicy to run the report.
>>
>> Given the stats from Protect the two backup servers are reporting
>> different occupancy figures which suggests to me we may have some
>> inconsistencies. (We're talking a 1/2 TB difference between the two
>> servers according to Protect Query Occupancy) but I'm aware that
>> Protect's figures are estimates and not always accurate.
>>
>> Verify the backup is always a good plan even if your 101% sure its
>> correct anyway (and I'm not maybe). (Your backups are only as good as
>> the last time you recalled them and all that)
>>
>> We could use the shadow database. As this looks to be what Storage
>> Archive does to say yes this file is backed up before it is archived.
>>
>> Does anyone know the format of the shadow database, which fields are
>> which, as I think knowing the format might allow us to at least know
>> what the differences; and increase our confidence in the backup.
>>
>> Thanks in advance.
>>
>> Peter Childs
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at gpfsug.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org

-- 
Stephan Graf
#GernePerDu
HPC, Cloud and Data Systems and Services
Jülich Supercomputing Centre

Phone:  +49-2461-61-6578
Fax:    +49-2461-61-6656
E-mail: st.graf at fz-juelich.de
WWW:    http://www.fz-juelich.de/jsc/
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
Forschungszentrum Jülich GmbH
52425 Jülich
Sitz der Gesellschaft: Jülich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Stefan Müller
Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende),
Dr. Stephanie Bauer (stellv. Vorsitzende)
Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6199 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20260116/f5934a98/attachment-0001.p7s>