[gpfsug-discuss] Capacity pool filling

[Buterbaugh, Kevin L]

Hi Uwe,

Thanks for your response.

So our restore software lays down the metadata first, then the data.  While it has no specific knowledge of the extended attributes, it does back them up and restore them.  So the only explanation that makes sense to me is that since the inode for the file says that the file should be in the gpfs23capacity pool, the data gets written there.

Right now I don’t have time to do an analysis of the “live” version of a fileset and the “restored” version of that same fileset to see if the placement of the files matches up.  My quick and dirty checks seem to show files getting written to all 3 pools.  Unfortunately, we have no way to tell our tape software to ignore files from the gpfs23capacity pool (and we’re aware that we won’t need those files).  We’ve also determined that it is actually quicker to tell our tape system to restore all files from a fileset than to take the time to tell it to selectively restore only certain files … and the same amount of tape would have to be read in either case.

Our SysAdmin who is primary on tape backup and restore was going on vacation the latter part of the week, so he decided to be helpful and just queue up all the restores to run one right after the other.  We didn’t realize that, so we are solving our disk space issues by slowing down the restores until we can run more instances of the script that replaces the corrupted files and deletes the unneeded restored files.

Thanks again…

Kevin

> On Jun 7, 2018, at 1:34 PM, Uwe Falke <UWEFALKE at de.ibm.com> wrote:
> 
>> However, I took a look in one of the restore directories under 
>> /gpfs23/ RESTORE using mmlsattr and I see files in all 3 pools! 
> 
> 
>> So ? I don?t think GPFS is doing this but the next thing I am 
>> going to do is follow up with our tape software vendor ? I bet 
>> they preserve the pool attribute on files and - like Jaime said - 
>> old stuff is therefore hitting the gpfs23capacity pool.
> 
> Hm, then the backup/restore must be doing very funny things. Usually, GPFS 
> should rule the 
> placement of new files, and I assume that a restore of a file, in 
> particular under a different name, 
> creates a new file. So, if your backup tool does override that GPFS 
> placement, it must be very 
> intimate with Scale :-). 
> I'd do some list scans of the capacity pool just to see what the files 
> appearing there from tape have in common. 
> If it's really that these files' data were on the capacity pool at the 
> last backup, they should not be affected by your dead NSD and a restore is 
> in vain anyway.
> 
> If that doesn't help or give no clue, then, if the data pool has some more 
> free  space, you might try to run an upward/backward migration from 
> capacity to data . 
> 
> And, yeah, as GPFS tends to stripe over all NSDs, all files in data large 
> enough plus some smaller ones would have data on your broken NSD. That's 
> the drawback of parallelization.
> Maybe you'd ask the storage vendor whether they supply some more storage 
> for the fault of their (redundant?) device to alleviate your current 
> storage shortage ?
> 
> Mit freundlichen Grüßen / Kind regards
> 
> 
> Dr. Uwe Falke
> 
> IT Specialist
> High Performance Computing Services / Integrated Technology Services / 
> Data Center Services
> -------------------------------------------------------------------------------------------------------------------------------------------
> IBM Deutschland
> Rathausstr. 7
> 09111 Chemnitz
> Phone: +49 371 6978 2165
> Mobile: +49 175 575 2877
> E-Mail: uwefalke at de.ibm.com
> -------------------------------------------------------------------------------------------------------------------------------------------
> IBM Deutschland Business & Technology Services GmbH / Geschäftsführung: 
> Thomas Wolter, Sven Schooß
> Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, 
> HRB 17122 
> 
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cacad30699025407bc67b08d5cca54bca%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636639932669887596&sdata=vywTFbG4O0lquAIAVfa0csdC0HtpvfhY8%2FOjqm98fxI%3D&reserved=0