<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> We have about 760-ish independent filesets. As was mentioned before in a reply, this allows for individual fileset snapshotting, and running on different TSM servers. We maintain a puppet-managed list that we use to divide up the filesets,
. automation helps us round-robin new filesets across the 4 backup servers as they are added to attempt to balance things somewhat. We maintain 7 days of snapshots on the filesystem we backup, and no snapshots or backups on our scratch space.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We hand out the mmbackups to 4 individual TSM backup clients which do both our daily mmbackup, and NetApp snappdiff backups for our user home directories as well. Those feed to another 4 TSM servers doing the tape migrations.
<o:p></o:p></p>
<p class="MsoNormal">We’re sitting on about ~20P of disk at this time and we’re (very) roughly 50% occupied. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">One of our challenges recently was re-balancing all this for remote Disaster Recovery/Replication. We ended up using colocation groups of the filesets in Spectrum Protect/TSM. While scaling backup infrastructure can be hard, balancing
hundreds of Wildly differing filesets can be just as challenging. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m happy to talk about these kinds of things here, or offline. Drop me a line if you have additional questions.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Ed Wahl<o:p></o:p></p>
<p class="MsoNormal">Ohio Supercomputer Center<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org>
<b>On Behalf Of </b>Christian Petersson<br>
<b>Sent:</b> Wednesday, September 6, 2023 5:45 AM<br>
<b>To:</b> gpfsug main discussion list <gpfsug-discuss@gpfsug.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] mmbackup feature request<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Just a follow up question, how do you backup multiple filesets? We have a 50 filesets to backup, at the moment do we have a text file that contains all of them
and we run a for loop. But that is not at all scalable. Is it any other ways that
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Just a follow up question, how do you backup multiple filesets? <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">We have a 50 filesets to backup, at the moment do we have a text file that contains all of them and we run a for loop. But that is not at all scalable. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Is it any other ways that are much better?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">/Christian <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">ons 6 sep. 2023 kl. 11:35 skrev Marcus Koenig <<a href="mailto:marcus@koenighome.de">marcus@koenighome.de</a>>:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">I'm using this one liner to get the progress<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">grep 'mmbackup:Backup job finished'|cut -d ":" -f 6|awk '{print $1}'|awk '{s+=$1}END{print s}'<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">That can be compared to the files identified during the scan.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, 6 Sept 2023, 21:29 Stephan Graf, <<a href="mailto:st.graf@fz-juelich.de" target="_blank">st.graf@fz-juelich.de</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Hi<br>
<br>
I think it should be possible because mmbackup know, how many files are <br>
to be backed up, which have been already processed and how many are <br>
still to go.<br>
<br>
BTW it would also be nice to have an option in mmbackup to generate <br>
machine readable log file like JSON or CSV.<br>
<br>
But the right way to ask for a new feature or to look if there is <br>
already a request open is the IBM IDEA portal:<br>
<br>
<a href="https://urldefense.com/v3/__https:/ideas.ibm.com__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHIMIKyXsk$" target="_blank">https://ideas.ibm.com</a><br>
<br>
Stephan<br>
<br>
On 9/6/23 11:02, Jonathan Buzzard wrote:<br>
> <br>
> Would it be possible to have the mmbackup output display the percentage <br>
> output progress when backing up files?<br>
> <br>
> So at the top we you see something like this<br>
> <br>
> Tue Sep 5 23:13:35 2023 mmbackup:changed=747204, expired=427702, <br>
> unsupported=0 for server [XXXX]<br>
> <br>
> Then after it does the expiration you see during the backup lines like<br>
> <br>
> Wed Sep 6 02:43:53 2023 mmbackup:Backing up files: 527024 backed up, <br>
> 426018 expired, 4408 failed. (Backup job exit with 4)<br>
> <br>
> It would IMHO be helpful if it looked like<br>
> <br>
> Wed Sep 6 02:43:53 2023 mmbackup:Backing up files: 527024 (70.5%) <br>
> backed up, 426018 (100%) expired, 4408 failed. (Backup job exit with 4)<br>
> <br>
> Just based on the number of files. Though as I look at it now I am <br>
> curious about the discrepancy in the number of files expired, given that <br>
> the expiration stage allegedly concluded with no errors?<br>
> <br>
> Tue Sep 5 23:21:49 2023 mmbackup:Completed policy expiry run with 0 <br>
> policy errors, 0 files failed, 0 severe errors, returning rc=0.<br>
> Tue Sep 5 23:21:49 2023 mmbackup:Policy for expiry returned 0 Highest <br>
> TSM error 0<br>
> <br>
> <br>
> <br>
> JAB.<br>
> <br>
<br>
-- <br>
Stephan Graf<br>
Juelich Supercomputing Centre<br>
<br>
Phone: +49-2461-61-6578<br>
Fax: +49-2461-61-6656<br>
E-mail: <a href="mailto:st.graf@fz-juelich.de" target="_blank">st.graf@fz-juelich.de</a><br>
WWW: <a href="https://urldefense.com/v3/__http:/www.fz-juelich.de/jsc/__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHIoADZPpY$" target="_blank">
http://www.fz-juelich.de/jsc/</a><br>
---------------------------------------------------------------------------------------------<br>
---------------------------------------------------------------------------------------------<br>
Forschungszentrum Juelich GmbH<br>
52425 Juelich<br>
Sitz der Gesellschaft: Juelich<br>
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498<br>
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke<br>
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),<br>
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,<br>
Prof. Dr. Frauke Melchior<br>
---------------------------------------------------------------------------------------------<br>
---------------------------------------------------------------------------------------------<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="https://urldefense.com/v3/__http:/gpfsug.org__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHIftznjDE$" target="_blank">
gpfsug.org</a><br>
<a href="https://urldefense.com/v3/__http:/gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHI-cdFpMc$" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="https://urldefense.com/v3/__http:/gpfsug.org__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHIftznjDE$" target="_blank">
gpfsug.org</a><br>
<a href="https://urldefense.com/v3/__http:/gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org__;!!KGKeukY!03ECSdA7lzhWsifFSkK9t1YtNYvDB89pvj-eJrh4gWV9IYpYH61rCBiaASdvtsHkekKlW5pqQriFk_mv-wHI-cdFpMc$" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</body>
</html>