[gpfsug-discuss] Spectrum Protect and disk pools

Alec.Effrat at wellsfargo.com Alec.Effrat at wellsfargo.com
Mon Jan 4 17:30:39 GMT 2021


I am not sure what platform you run on but for AIX with a fully virtualized LPAR we needed to enable "mtu_bypass" on the en device that was used for our backups.  Prior to this setting we could not exceed 250 MB/s on our 10G interface, after that we run at 1.6GB/s solid per 10G virtual adapter, fueled by Spectrum Scale and a different backup engine.   

We did lose a lot of sleep trying to figure this one out, but are very pleased with the end result.

Alec Effrat
SAS Lead, AVP
Business Intelligence Competency Center
SAS Administration
Cell 949-246-7713
alec.effrat at wellsfargo.com

-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of Jonathan Buzzard
Sent: Monday, January 4, 2021 8:24 AM
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] Spectrum Protect and disk pools

On 04/01/2021 12:21, Simon Thompson wrote:
> CAUTION: This email originated outside the University. Check before 
> clicking links or attachments.
> 
> Hi All,
> 
> We use Spectrum Protect (TSM) to backup our Scale filesystems. We have 
> the backup setup to use multiple nodes with the PROXY node function 
> turned on (and to some extent also use multiple target servers).
> 
> This all feels like it is nice and parallel, on the TSM servers, we 
> have disk pools for any “small” files to drop into (I think we set 
> anything smaller than 20GB) to prevent lots of small files stalling 
> tape drive writes.
> 
> Whilst digging into why we have slow backups at times, we found that 
> the disk pool empties with a single thread (one drive). And looking at the docs:
> 
> https://www.ibm.com/support/pages/concurrent-migration-processes-and-c
> onstraints 
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
> .ibm.com%2Fsupport%2Fpages%2Fconcurrent-migration-processes-and-constr
> aints&data=04%7C01%7Cjonathan.buzzard%40strath.ac.uk%7C99158004dad04c7
> 9a58808d8b0ab39b8%7C631e0763153347eba5cd0457bee5944e%7C0%7C0%7C6374535
> 96745356438%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZPUkTB5Vy5S0%2BL67neMp4C
> 1lxIuphMS5HuTkBYcmcMU%3D&reserved=0>
> 
> This implies that we are limited to the number of client nodes stored 
> in the pool. i.e. because we have one node and PROXY nodes, we are 
> essentially limited to a single thread streaming out of the disk pool 
> when full.
> 
> Have we understood this correctly as if so, this appears to make the 
> whole purpose of PROXY nodes sort of pointless if you have lots of 
> small files. Or is there some other setting we should be looking at to 
> increase the number of threads when the disk pool is emptying? (The 
> disk pool itself has Migration Processes: 6)
> 

I have found in the past that the speed of the disk pool can make a large difference. That is a RAID5/6 of 7200RPM drives was inadequate and there was a significant boost in backup in moving to 15k RPM disks. Also your DB really needs to be on SSD, again this affords a large boost in backup speed.

The other rule of thumb I have always worked with is that the disk pool should be sized for the daily churn. That is your backup should disappear into the disk pool and then when the backup is finished you can then spit the disk pool out to the primary and copy pools.

If you are needing to drain the disk pool mid backup your disk pool is too small.

TL;DR your TSM disks (DB and disk pool) need to be some of the best storage you have to maximize backup speed.

JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG _______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


More information about the gpfsug-discuss mailing list