[gpfsug-discuss] mmbackup feature request

[Skylar Thompson]

We have a hacky way of doing this by getting the size of the update and
expire lists, i.e.:

wc -l /gpfs/fs1/.mmbackupCfg/updatedFiles/.list.1.node-name

And also each index number of the chunked file lists that dsmc is
processing, for instance:

-filelist=/gpfs/fs1/.mmbackupCfg/mmbackupChanged.ix.13589.0424C05C.465.fs1

Which shows index 465. The final bit of information you need is the
MaxFiles setting of mmbackup, which is calculated automatically by default
but is visible in mmapplypolicy and tsapolicy command lines in /proc in the
-B value (ours is 32768).

Once you have all of that information, you can calculate the number of file
lists that will be processed across all the backup nodes by dividing the
length of the updatedFiles and expireFiles lists by the MaxFiles setting to
get the total number of file lists that will be processed, and the number
of file lists in by adding up the index numbers from all of your backup
nodes.

That said, if there were an easy-to-parse option in the logs, we would
certainly use it. :)

On Wed, Sep 06, 2023 at 10:02:17AM +0100, Jonathan Buzzard wrote:
> 
> Would it be possible to have the mmbackup output display the percentage
> output progress when backing up files?
> 
> So at the top we you see something like this
> 
> Tue Sep  5 23:13:35 2023 mmbackup:changed=747204, expired=427702,
> unsupported=0 for server [XXXX]
> 
> Then after it does the expiration you see during the backup lines like
> 
> Wed Sep  6 02:43:53 2023 mmbackup:Backing up files: 527024 backed up, 426018
> expired, 4408 failed. (Backup job exit with 4)
> 
> It would IMHO be helpful if it looked like
> 
> Wed Sep  6 02:43:53 2023 mmbackup:Backing up files: 527024 (70.5%) backed
> up, 426018 (100%) expired, 4408 failed. (Backup job exit with 4)
> 
> Just based on the number of files. Though as I look at it now I am curious
> about the discrepancy in the number of files expired, given that the
> expiration stage allegedly concluded with no errors?
> 
> Tue Sep  5 23:21:49 2023 mmbackup:Completed policy expiry run with 0 policy
> errors, 0 files failed, 0 severe errors, returning rc=0.
> Tue Sep  5 23:21:49 2023 mmbackup:Policy for expiry returned 0 Highest TSM
> error 0
> 
> 
> 
> JAB.
> 
> -- 
> Jonathan A. Buzzard                         Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org__;!!K-Hz7m0Vt54!kKHB7ax8P1MfIPjQ35h55FpbmMkyaHMoKFgvcA6DjcVkZpEfeSXgkvXueM9LKJ0wWDND1YATiCfht16ed_PWfVNKevmKpYE$

-- 
-- Skylar Thompson (skylar2 at u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His