<div><font size=2 face="sans-serif">there're multiple dependencies , the
performance of MD scan is related to</font><br><font size=2 face="sans-serif">as a rule of thumb... </font><br><font size=2 face="sans-serif">the total amount of IOPS you need to
scan your MD is highly dependent on the metadata blocksize, inode size
(assuming default 4K ) ( and the total number Inodes.. ;-) ) </font><br><font size=2 face="sans-serif">the time it takes to answer these IOs
depends on your backend(s) , and ... .. </font><br><font size=2 face="sans-serif">the parallelism and the node's hardware
resource and finally the network connectivity (latency, bandwidth)
</font><br><br><font size=2 face="sans-serif">to give some directions... </font><br><font size=2 face="sans-serif">we even have clusters, using regular
(old and spinning) drives , and 're able to scan > 200 mio files within
< 15 minutes.. </font><br><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Knister, Aaron
S. (GSFC-606.2)[COMPUTER SCIENCE CORP]" <aaron.s.knister@nasa.gov></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org>, "gpfsug
main discussion list" <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">08/31/2016 06:01 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
*New* IBM Spectrum Protect Whitepaper "Petascale Data Protection"</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=3 color=#004080>Just want to add on to one of the points
Sven touched on regarding metadata HW. We have a modest SSD infrastructure
for our metadata disks and we can scan 500M inodes in parallel in about
5 hours if my memory serves me right (and I believe we could go faster
if we really wanted to). I think having solid metadata disks (no pun intended)
will really help with scan times. </font><br><font size=3><br></font><p><font size=2 face="Helvetica"><b>From:</b> Sven Oehme<b><br>Sent:</b> 8/30/16, 7:25 PM<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> Re: [gpfsug-discuss] *New* IBM Spectrum Protect Whitepaper
"Petascale Data Protection"</font><p><font size=3>so lets start with some simple questions. </font><br><br><font size=3>when you say mmbackup takes ages, what version of gpfs
code are you running ? </font><br><font size=3>how do you execute the mmbackup command ? exact parameters
would be useful . </font><br><font size=3>what HW are you using for the metadata disks ? </font><br><font size=3>how much capacity (df -h) and how many inodes (df -i)
do you have in the filesystem you try to backup ?</font><br><br><font size=3>sven</font><br><br><br><font size=3>On Tue, Aug 30, 2016 at 3:02 PM, Lukas Hejtmanek <</font><a href=mailto:xhejtman@ics.muni.cz><font size=3 color=blue><u>xhejtman@ics.muni.cz</u></font></a><font size=3>>
wrote:</font><br><font size=3>Hello,<br><br>On Mon, Aug 29, 2016 at 09:20:46AM +0200, Frank Kraemer wrote:<br>> Find the paper here:<br>><br>> </font><a href="https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/Petascale%20Data%20Protection" target=_BLANK><font size=3 color=blue><u>https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/Petascale%20Data%20Protection</u></font></a><font size=3><br><br>thank you for the paper, I appreciate it.<br><br>However, I wonder whether it could be extended a little. As it has the
title<br>Petascale Data Protection, I think that in Peta scale, you have to deal
with<br>millions (well rather hundreds of millions) of files you store in and this
is<br>something where TSM does not scale well.<br><br>Could you give some hints:<br><br>On the backup site:<br>mmbackup takes ages for:<br>a) scan (try to scan 500M files even in parallel)<br>b) backup - what if 10 % of files get changed - backup process can be blocked<br>several days as mmbackup cannot run in several instances on the same file<br>system, so you have to wait until one run of mmbackup finishes. How long
could<br>it take at petascale?<br><br>On the restore site:<br>how can I restore e.g. 40 millions of file efficiently? dsmc restore '/path/*'<br>runs into serious troubles after say 20M files (maybe wrong internal<br>structures used), however, scanning 1000 more files takes several minutes<br>resulting the dsmc restore never reaches that 40M files.<br><br>using filelists the situation is even worse. I run dsmc restore -filelist<br>with a filelist consisting of 2.4M files. Running for *two* days without<br>restoring even a single file. dsmc is consuming 100 % CPU.<br><br>So any hints addressing these issues with really large number of files
would<br>be even more appreciated.</font><font size=3 color=#8f8f8f><br><br>--<br>Lukáš Hejtmánek<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href=http://spectrumscale.org/ target=_BLANK><font size=3 color=blue><u>spectrumscale.org</u></font></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target=_BLANK><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><br><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br></div><BR>