[gpfsug-discuss] mmbackup file selections
Skylar Thompson
skylar2 at uw.edu
Mon Jan 24 15:36:31 GMT 2022
Hi Paul,
Did you look for dot files? At least for us on 5.0.5 there's a
.list.1.<tsm-node> file while the backups are running:
/gpfs/grc6/.mmbackupCfg/updatedFiles/:
-r-------- 1 root nickers 6158526821 Jan 23 18:28 .list.1.gpfs-grc6
/gpfs/grc6/.mmbackupCfg/expiredFiles/:
-r-------- 1 root nickers 85862211 Jan 23 18:28 .list.1.gpfs-grc6
On Mon, Jan 24, 2022 at 02:31:54PM +0000, Paul Ward wrote:
> Those directories are empty
>
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>
> [A picture containing drawing Description automatically generated]
>
> From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of IBM Spectrum Scale
> Sent: 22 January 2022 00:35
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Cc: gpfsug-discuss-bounces at spectrumscale.org
> Subject: Re: [gpfsug-discuss] mmbackup file selections
>
>
> Hi Paul,
>
> Instead of calculating *.ix.* files, please look at a list file in these directories.
>
> updatedFiles : contains a file that lists all candidates for backup
> statechFiles : cantains a file that lists all candidates for meta info update
> expiredFiles : cantains a file that lists all candidates for expiration
>
> Regards, The Spectrum Scale (GPFS) team
>
> ------------------------------------------------------------------------------------------------------------------
>
> If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
>
>
> [Inactive hide details for "Paul Ward" ---01/21/2022 09:38:49 AM---Thank you Right in the command line seems to have worked.]"Paul Ward" ---01/21/2022 09:38:49 AM---Thank you Right in the command line seems to have worked.
>
> From: "Paul Ward" <p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>>
> To: "gpfsug main discussion list" <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
> Cc: "gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>" <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>>
> Date: 01/21/2022 09:38 AM
> Subject: [EXTERNAL] Re: [gpfsug-discuss] mmbackup file selections
> Sent by: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
>
> ________________________________
>
>
>
> Thank you Right in the command line seems to have worked. At the end of the script I now copy the contents of the .mmbackupCfg folder to a date stamped logging folder Checking how many entries in these files compared to the Summary: ???????ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> Thank you
>
> Right in the command line seems to have worked.
> At the end of the script I now copy the contents of the .mmbackupCfg folder to a date stamped logging folder
>
> Checking how many entries in these files compared to the Summary:
> wc -l mmbackup*
> 188 mmbackupChanged.ix.155513.6E9E8BE2.1.nhmfsa
> 47 mmbackupChanged.ix.219901.8E89AB35.1.nhmfsa
> 188 mmbackupChanged.ix.37893.EDFB8FA7.1.nhmfsa
> 40 mmbackupChanged.ix.81032.78717A00.1.nhmfsa
> 2 mmbackupExpired.ix.78683.2DD25239.1.nhmfsa
> 141 mmbackupStatech.ix.219901.8E89AB35.1.nhmfsa
> 148 mmbackupStatech.ix.81032.78717A00.1.nhmfsa
> 754 total
> From Summary
> Total number of objects inspected: 755
> I can live with a discrepancy of 1.
>
> 2 mmbackupExpired.ix.78683.2DD25239.1.nhmfsa
> From Summary
> Total number of objects expired: 2
> That matches
>
> wc -l mmbackupC* mmbackupS*
> 188 mmbackupChanged.ix.155513.6E9E8BE2.1.nhmfsa
> 47 mmbackupChanged.ix.219901.8E89AB35.1.nhmfsa
> 188 mmbackupChanged.ix.37893.EDFB8FA7.1.nhmfsa
> 40 mmbackupChanged.ix.81032.78717A00.1.nhmfsa
> 141 mmbackupStatech.ix.219901.8E89AB35.1.nhmfsa
> 148 mmbackupStatech.ix.81032.78717A00.1.nhmfsa
> 752 total
> Summary:
> Total number of objects backed up: 751
>
> A difference of 1 I can live with.
>
> What does Statech stand for?
>
> Just this to sort out:
> Total number of objects failed: 1
> I will add:
> --tsm-errorlog TSMErrorLogFile
>
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>
> [A picture containing drawing Description automatically generated]
>
> From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> On Behalf Of IBM Spectrum Scale
> Sent: 19 January 2022 15:09
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
> Cc: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
> Subject: Re: [gpfsug-discuss] mmbackup file selections
>
>
> This is to set environment for mmbackup.
> If mmbackup is invoked within a script, you can set "export DEBUGmmbackup=2" right above mmbackup command.
> e.g) in your script
> ....
> export DEBUGmmbackup=2
> mmbackup ....
>
> Or, you can set it in the same command line like
> DEBUGmmbackup=2 mmbackup ....
>
> Regards, The Spectrum Scale (GPFS) team
>
> ------------------------------------------------------------------------------------------------------------------
> If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
>
> [Inactive hide details for "Paul Ward" ---01/19/2022 06:04:03 AM---Thank you. We run a script on all our nodes that checks to se]"Paul Ward" ---01/19/2022 06:04:03 AM---Thank you. We run a script on all our nodes that checks to see if they are the cluster manager.
>
> From: "Paul Ward" <p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>>
> To: "gpfsug main discussion list" <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
> Cc: "gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>" <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>>
> Date: 01/19/2022 06:04 AM
> Subject: [EXTERNAL] Re: [gpfsug-discuss] mmbackup file selections
> Sent by: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
>
> ________________________________
>
>
>
>
> Thank you. We run a script on all our nodes that checks to see if they are the cluster manager. If they are, then they take responsibility to start the backup script. The script then randomly selects one of the available backup nodes and uses ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> Thank you.
>
> We run a script on all our nodes that checks to see if they are the cluster manager.
> If they are, then they take responsibility to start the backup script.
> The script then randomly selects one of the available backup nodes and uses dsmsh mmbackup on it.
>
> Where does this command belong?
> I have seen it listed as a export command, again where should that be run ? on all backup nodes, or all nodes?
>
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>
> [A picture containing drawing Description automatically generated]
>
> From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> On Behalf Of IBM Spectrum Scale
> Sent: 18 January 2022 22:54
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
> Cc: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
> Subject: Re: [gpfsug-discuss] mmbackup file selections
>
> Hi Paul,
>
> If you run mmbackup with "DEBUGmmbackup=2", it keeps all working files even after successful backup. They are available at MMBACKUP_RECORD_ROOT (default is FSroot or FilesetRoot directory).
> In .mmbackupCfg directory, there are 3 directories:
> updatedFiles : contains a file that lists all candidates for backup
> statechFiles : cantains a file that lists all candidates for meta info update
> expiredFiles : cantains a file that lists all candidates for expiration
>
>
> Regards, The Spectrum Scale (GPFS) team
>
> ------------------------------------------------------------------------------------------------------------------
> If your query concerns a potential software error in Spectrum Scale (GPFS) and you have an IBM software maintenance contract please contact 1-800-237-5511 in the United States or your local IBM Service Center in other countries.
>
> [Inactive hide details for "Paul Ward" ---01/18/2022 11:56:40 AM---Hi, I am trying to work out what files have been sent to back]"Paul Ward" ---01/18/2022 11:56:40 AM---Hi, I am trying to work out what files have been sent to backup using mmbackup.
>
> From: "Paul Ward" <p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>>
> To: "gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>" <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
> Date: 01/18/2022 11:56 AM
> Subject: [EXTERNAL] [gpfsug-discuss] mmbackup file selections
> Sent by: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
>
> ________________________________
>
>
>
>
>
> Hi, I am trying to work out what files have been sent to backup using mmbackup. I have increased the -L value from 3 up to 6 but only seem to see the files that are in scope, not the ones that are selected. I can see the three file lists generated ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> Hi,
>
> I am trying to work out what files have been sent to backup using mmbackup.
> I have increased the -L value from 3 up to 6 but only seem to see the files that are in scope, not the ones that are selected.
>
> I can see the three file lists generated during a backup, but can?t seem to find a list of what files were backed up.
>
> It should be the diff of the shadow and shadow-old, but the wc -l of the diff doesn?t match the number of files in the backup summary.
> Wrong assumption?
>
> Where should I be looking ? surely it shouldn?t be this hard to see what files are selected?
>
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk<mailto:p.ward at nhm.ac.uk>
> [A picture containing drawing Description automatically generated]
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=04%7C01%7Cp.ward%40nhm.ac.uk%7C17acea2a1b4944dc8daa08d9dd3f1fea%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637784085889014099%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=jJykq3GwRIajX24bWsBuk6ydadeiumBiSLZeKnMdiZw%3D&reserved=0>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=04%7C01%7Cp.ward%40nhm.ac.uk%7C17acea2a1b4944dc8daa08d9dd3f1fea%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637784085889014099%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=jJykq3GwRIajX24bWsBuk6ydadeiumBiSLZeKnMdiZw%3D&reserved=0>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=04%7C01%7Cp.ward%40nhm.ac.uk%7C17acea2a1b4944dc8daa08d9dd3f1fea%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C637784085889014099%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=jJykq3GwRIajX24bWsBuk6ydadeiumBiSLZeKnMdiZw%3D&reserved=0>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
--
-- Skylar Thompson (skylar2 at u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His
More information about the gpfsug-discuss
mailing list