[gpfsug-discuss] gpfsug-discuss Digest, Vol 51, Issue 9

Jez Tucker jez.tucker at gpfsug.org
Mon Apr 11 16:31:52 BST 2016 

Dominic,

   Speculatively, when is TSM converting from DMAPI to Light Weight Events?
Is there an up-to-date slide share we can put on the UG website 
regarding the 7.1.11 / public roadmap?

Jez


On 11/04/16 16:26, Dominic Mueller-Wicke01 wrote:
>
> Spectrum Protect backup (under the hood of mmbackup) and Spectrum 
> Protect for Space Management (HSM) can be combined on the same data. 
> There are some valuable integration topics between the products that 
> can reduce the overall network traffic if using backup and HSM on the 
> same files. With the combination of the products you have the ability 
> to free file system space from cold data and migrate them out to tape 
> and to have several versions of frequently used files in backup in the 
> same file system.
>
> Greetings, Dominic.
>
> ______________________________________________________________________________________________________________
> Dominic Mueller-Wicke | IBM Spectrum Protect Development | Technical 
> Lead | +49 7034 64 32794 | dominic.mueller at de.ibm.com
>
> Vorsitzende des Aufsichtsrats: Martina Koederitz; Geschäftsführung: 
> Dirk Wittkopp
> Sitz der Gesellschaft: Böblingen; Registergericht: Amtsgericht 
> Stuttgart, HRB 243294
>
> Inactive hide details for gpfsug-discuss-request---11.04.2016 
> 17:11:55---Send gpfsug-discuss mailing list submissions to 
> gpfsugpfsug-discuss-request---11.04.2016 17:11:55---Send 
> gpfsug-discuss mailing list submissions to 
> gpfsug-discuss at spectrumscale.org
>
> From: gpfsug-discuss-request at spectrumscale.org
> To: gpfsug-discuss at spectrumscale.org
> Date: 11.04.2016 17:11
> Subject: gpfsug-discuss Digest, Vol 51, Issue 9
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
>
> ------------------------------------------------------------------------
>
>
>
> Send gpfsug-discuss mailing list submissions to
> gpfsug-discuss at spectrumscale.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> or, via email, send a message with subject or body 'help' to
> gpfsug-discuss-request at spectrumscale.org
>
> You can reach the person managing the list at
> gpfsug-discuss-owner at spectrumscale.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of gpfsug-discuss digest..."
> Today's Topics:
>
>   1. backup and disaster recovery solutions (Damir Krstic)
>   2. Re: backup and disaster recovery solutions (Jaime Pinto)
>   3. Re: backup and disaster recovery solutions (Jonathan Buzzard)
>   4. Re: backup and disaster recovery solutions (Marc A Kaplan)
>
> ----- Message from Damir Krstic <damir.krstic at gmail.com> on Mon, 11 
> Apr 2016 13:15:30 +0000 -----
> *To:*
> 	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> *Subject:*
> 	[gpfsug-discuss] backup and disaster recovery solutions
>
> We have implemented 1.5PB ESS solution recently in our HPC 
> environment. Today we are kicking of backup and disaster recovery 
> discussions so I was wondering what everyone else is using for their 
> backup?
>
> In our old storage environment we simply rsync-ed home and software 
> directories and projects were not backed up.
>
> With ESS we are looking for more of a GPFS based backup solution - 
> something to tape possibly and also something that will have life 
> cycle feature - so if the file is not touched for number of days, it's 
> moved to a tape (something like LTFS).
>
> Thanks in advance.
>
> DAmir
> ----- Message from Jaime Pinto <pinto at scinet.utoronto.ca> on Mon, 11 
> Apr 2016 10:34:54 -0400 -----
> *To:*
> 	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>, Damir 
> Krstic <damir.krstic at gmail.com>
> *Subject:*
> 	Re: [gpfsug-discuss] backup and disaster recovery solutions
>
> Do you want backups or periodic frozen snapshots of the file system?
>
> Backups can entail some level of version control, so that you or
> end-users can get files back on certain points in time, in case of
> accidental deletions. Besides 1.5PB is a lot of material, so you may
> not want to take full snapshots that often. In that case, a
> combination of daily incremental backups using TSM with GPFS's
> mmbackup can be a good option. TSM also does a very good job at
> controlling how material is distributed across multiple tapes, and
> that is something that requires a lot of micro-management if you want
> a home grown solution of rsync+LTFS.
>
> On the other hand, you could use gpfs built-in tools such a
> mmapplypolicy to identify candidates for incremental backup, and send
> them to LTFS. Just more micro management, and you may have to come up
> with your own tool to let end-users restore their stuff, or you'll
> have to act on their behalf.
>
> Jaime
>
>
>
>
> Quoting Damir Krstic <damir.krstic at gmail.com>:
>
> > We have implemented 1.5PB ESS solution recently in our HPC environment.
> > Today we are kicking of backup and disaster recovery discussions so 
> I was
> > wondering what everyone else is using for their backup?
> >
> > In our old storage environment we simply rsync-ed home and software
> > directories and projects were not backed up.
> >
> > With ESS we are looking for more of a GPFS based backup solution -
> > something to tape possibly and also something that will have life cycle
> > feature - so if the file is not touched for number of days, it's 
> moved to a
> > tape (something like LTFS).
> >
> > Thanks in advance.
> >
> > DAmir
> >
>
>
>
>
>
>
>          ************************************
>           TELL US ABOUT YOUR SUCCESS STORIES
> http://www.scinethpc.ca/testimonials
>          ************************************
> ---
> Jaime Pinto
> SciNet HPC Consortium  - Compute/Calcul Canada
> www.scinet.utoronto.ca - www.computecanada.org
> University of Toronto
> 256 McCaul Street, Room 235
> Toronto, ON, M5T1W5
> P: 416-978-2755
> C: 416-505-1477
>
> ----------------------------------------------------------------
> This message was sent using IMP at SciNet Consortium, University of 
> Toronto.
>
>
>
>
> ----- Message from Jonathan Buzzard <jonathan at buzzard.me.uk> on Mon, 
> 11 Apr 2016 16:02:45 +0100 -----
> *To:*
> 	gpfsug-discuss at spectrumscale.org
> *Subject:*
> 	Re: [gpfsug-discuss] backup and disaster recovery solutions
>
> On Mon, 2016-04-11 at 10:34 -0400, Jaime Pinto wrote:
> > Do you want backups or periodic frozen snapshots of the file system?
> >
> > Backups can entail some level of version control, so that you or
> > end-users can get files back on certain points in time, in case of
> > accidental deletions. Besides 1.5PB is a lot of material, so you may
> > not want to take full snapshots that often. In that case, a
> > combination of daily incremental backups using TSM with GPFS's
> > mmbackup can be a good option. TSM also does a very good job at
> > controlling how material is distributed across multiple tapes, and
> > that is something that requires a lot of micro-management if you want
> > a home grown solution of rsync+LTFS.
>
> Is there any other viable option other than TSM for backing up 1.5PB of
> data? All other backup software does not handle this at all well.
>
> > On the other hand, you could use gpfs built-in tools such a
> > mmapplypolicy to identify candidates for incremental backup, and send
> > them to LTFS. Just more micro management, and you may have to come up
> > with your own tool to let end-users restore their stuff, or you'll
> > have to act on their behalf.
> >
>
> I was not aware of a way of letting end users restore their stuff from
> *backup* for any of the major backup software while respecting the file
> system level security of the original file system. If you let the end
> user have access to the backup they can restore any file to any location
> which is generally not a good idea.
>
> I do have a concept of creating a read only Fuse mounted file system
> from a TSM point in time synthetic backup, and then using the shadow
> copy feature of Samba to enable restores using the "Previous Versions"
> feature of windows file manager.
>
> I got as far as getting a directory tree you could browse through but
> then had an enforced change of jobs and don't have access to a TSM
> server any more to continue development.
>
> Note if anyone from IBM is listening that would be a super cool feature.
>
>
> JAB.
>
> -- 
> Jonathan A. Buzzard                 Email: jonathan (at) buzzard.me.uk
> Fife, United Kingdom.
>
>
>
>
> ----- Message from "Marc A Kaplan" <makaplan at us.ibm.com> on Mon, 11 
> Apr 2016 11:11:24 -0400 -----
> *To:*
> 	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> *Subject:*
> 	Re: [gpfsug-discuss] backup and disaster recovery solutions
>
> Since you write "so if the file is not touched for number of days, 
> it's moved to a tape" -
> that is what we call the HSM feature. This is additional function 
> beyond backup. IBM has two implementations.
>
> (1) TSM/HSM now called IBM Spectrum Protect.
> _http://www-03.ibm.com/software/products/en/spectrum-protect-for-space-management_
>
> (2) HPSS _http://www.hpss-collaboration.org/_
>
> The GPFS (Spectrum Scale File System) policy feature supports both, so 
> that mmapplypolicy and GPFS policy rules can be used to perform
> accelerated metadata scans to identify which files should be migrated.
>
> Also, GPFS supports on-demand recall (on application reads) of data 
> from long term storage (tape) to GPFS storage (disk or SSD). See also 
> DMAPI.
>
>
>
> Marc A Kaplan
>
>
>
> From: Damir Krstic <damir.krstic at gmail.com>
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Date: 04/11/2016 09:16 AM
> Subject: [gpfsug-discuss] backup and disaster recovery solutions
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> ------------------------------------------------------------------------
>
>
>
> We have implemented 1.5PB ESS solution recently in our HPC 
> environment. Today we are kicking of backup and disaster recovery 
> discussions so I was wondering what everyone else is using for their 
> backup?
>
> In our old storage environment we simply rsync-ed home and software 
> directories and projects were not backed up.
>
> With ESS we are looking for more of a GPFS based backup solution - 
> something to tape possibly and also something that will have life 
> cycle feature - so if the file is not touched for number of days, it's 
> moved to a tape (something like LTFS).
>
> Thanks in advance.
>
> DAmir _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org_
> __http://gpfsug.org/mailman/listinfo/gpfsug-discuss_
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160411/9dc50c72/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160411/9dc50c72/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 21994 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160411/9dc50c72/attachment-0005.gif>

More information about the gpfsug-discuss mailing list