<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">I understand that this near linear performance is one of the differentiators of Spectrum Scale.<div>Others with more field experience than me might want to comment on how Lustre and other distributed filesystem perform as they approaches near full capacity.</div><div><br><div id="AppleMailSignature"><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);">Daniel</span></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);"><img alt="/spectrum_storage-banne" src="http://ausgsa.ibm.com/projects/t/tivoli_visual_design/public/2015/Spectrum-Storage/Email-signatures/Storage/spectrum_storage-banner.png" width="432.5" height="auto" style="width: 601px; height: 5px;"></span></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);"><br> </span></div><table border="0" cellpadding="0" cellspacing="0" style="font-family: sans-serif;"><tbody><tr style="word-wrap: break-word; max-width: 447.5px;"><td style="word-wrap: break-word; max-width: 447.5px; width: 201px; padding: 0cm;"><div style="margin-bottom: 0.0001pt; line-height: normal;"><img alt="Spectrum Scale Logo" src="http://ausgsa.ibm.com/projects/t/tivoli_visual_design/public/2015/Spectrum-Storage/Email-signatures/Storage/spectrum_scale-logo.png" style="width: 75px; height: 120px; float: left;"></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><font face=".SFUIDisplay"><span style="background-color: rgba(255, 255, 255, 0);"> </span></font></div></td><td style="word-wrap: break-word; max-width: 447.5px; width: 21px; padding: 0cm;"><font face=".SFUIDisplay"><span style="background-color: rgba(255, 255, 255, 0);"> </span></font></td><td style="word-wrap: break-word; max-width: 447.5px; width: 202px; padding: 0cm;"><div style="margin-bottom: 0.0001pt; line-height: normal;"><font face=".SFUIDisplay"><span style="background-color: rgba(255, 255, 255, 0);"><strong>Dr Daniel Kidger</strong> <br>IBM Technical Sales Specialist<br>Software Defined Solution Sales<br><br><a dir="ltr" href="tel:+%2044-7818%20522%20266" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1">+</a> <a dir="ltr" href="tel:+%2044-7818%20522%20266" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1">44-(0)7818 522 266</a> <br><a dir="ltr" href="mailto:daniel.kidger@uk.ibm.com" x-apple-data-detectors="true" x-apple-data-detectors-type="link" x-apple-data-detectors-result="2">daniel.kidger@uk.ibm.com</a></span></font></div></td></tr></tbody></table></div><div><br>On 7 Nov 2017, at 00:12, Carl <<a href="mailto:mutantllama@gmail.com">mutantllama@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">Thanks to all for the information. <div><br></div><div>Im happy to say that it is close to what I hoped would be the case. </div><div><br></div><div>Interesting to see the effect of the -n value. Reinforces the need to think about it and not go with the defaults.</div><div><br></div><div>Thanks again,</div><div><br></div><div>Carl.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 7 November 2017 at 03:18, Achim Rehor <span dir="ltr"><<a href="mailto:Achim.Rehor@de.ibm.com" target="_blank">Achim.Rehor@de.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font size="2" face="sans-serif">I have no practical experience on these
numbers, however, Peters experience below is matching what i learned from
Dan years ago.</font><br><br><font size="2" face="sans-serif">As long as the -n setting of the FS
(the number of nodes potentially mounting the fs) is more or less matching
the actual number of mounts, </font><br><font size="2" face="sans-serif">this 99.x % before degradation is expected.
If you are far off with that -n estimate, like having it set to 32, but
the actual number of mounts is in the thousands,</font><br><font size="2" face="sans-serif">then degradation happens earlier, since
the distribution of free blocks in the allocation maps is not matching
the actual setup as good as it could be. </font><br><br><font size="2" face="sans-serif">Naturally, this depends also on how
you do filling of the FS. If it is only a small percentage of the nodes,
doing the creates, then the distribution can </font><br><font size="2" face="sans-serif">be 'wrong' as well, and single nodes
run earlier out of allocation map space, and need to look for free blocks
elsewhere, costing RPC cycles and thus performance. </font><br><br><font size="2" face="sans-serif">Putting this in numbers seems quite
difficult ;) </font><br><br><font size="1" face="Arial"><br>Mit freundlichen Grüßen / Kind regards</font><p><font size="2" face="Arial"><b>Achim Rehor</b></font></p><p></p><table width="600" style="border-collapse:collapse"><tbody><tr height="8"><td width="600" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="center"><hr noshade=""></div></td></tr><tr height="8"><td width="600" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial"> </font></td></tr><tr height="8"><td width="496" colspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial">Software
Technical Support Specialist AIX/ Emea HPC Support</font></td><td width="103" rowspan="10" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="right"><_1_D95FF418D95FEE980059980B852581D0.gif></div></td></tr><tr height="8"><td width="496" colspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial">IBM
Certified Advanced Technical Expert - Power Systems with AIX</font></td></tr><tr height="8"><td width="496" colspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial">TSCC
Software Service, Dept. 7922</font></td></tr><tr height="8"><td width="496" colspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial">Global
Technology Services </font></td></tr><tr height="8"><td width="496" colspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="center"><hr noshade=""></div></td></tr><tr height="8"><td width="44" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial">Phone:</font></td><td width="195" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"><a href="tel:+49%207034%202747862" value="+4970342747862" target="_blank">+49-7034-274-7862</a></font></td><td width="257" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> IBM
Deutschland</font></td></tr><tr height="8"><td width="44" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial">E-Mail:</font></td><td width="195" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"><a href="mailto:Achim.Rehor@de.ibm.com" target="_blank">Achim.Rehor@de.ibm.com</a></font></td><td width="257" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> Am
Weiher 24</font></td></tr><tr height="8"><td width="44" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="195" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="257" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> 65451
Kelsterbach</font></td></tr><tr height="8"><td width="44" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="195" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="257" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3DAm-250D-2BWeiher-2B24-25C2-25A0-25C2-25A0-25C2-25A065451-250D-2BKelsterbach-25C2-25A0-25C2-25A0-25C2-25A0Germany-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=LcAQSuZfiGE8amC7zXd_vu0W02tmvaguHINvGR6UYTs&e=">Germany</a></font></td></tr><tr height="8"><td width="44" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="195" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td><td width="257" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#0060a0" face="Arial"> </font></td></tr><tr height="8"><td width="600" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="center"><hr noshade=""></div></td></tr><tr height="8"><td width="600" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial"> </font></td></tr><tr height="8"><td width="600" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#a2a2a2" face="Arial">IBM
Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin Jetter <br>Geschäftsführung: Martina Koederitz (Vorsitzende), Reinhard Reschke, Dieter
Scholz, Gregor Pillen, Ivo Koerner, Christian Noll <br>Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 14562 WEEE-Reg.-Nr. DE 99369940 </font></td></tr></tbody></table><br><p><font size="3"> </font><br><br><br><br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif">Peter Smith <<a href="mailto:peter.smith@framestore.com" target="_blank">peter.smith@framestore.com</a>></font><span class=""><br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif">gpfsug main discussion
list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.<wbr>org</a>></font><br></span><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">11/06/2017 09:17 AM</font><br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">Re: [gpfsug-discuss]
Performance of GPFS when filesystem is almost full</font><br><font size="1" color="#5f5f5f" face="sans-serif">Sent by:
</font><font size="1" face="sans-serif"><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@<wbr>spectrumscale.org</a></font><br></p><hr noshade=""><span class=""><br><br><br><tt><font size="3">Hi Carl.<br></font></tt><br><tt><font size="3">When we commissioned our system we ran an NFS stress
tool, and filled the system to the top.<br></font></tt><br><tt><font size="3">No performance degradation was seen until it was 99.7%
full.<br></font></tt><br><tt><font size="3">I believe that after this point it takes longer to
find free blocks to write to.<br></font></tt><br><tt><font size="3">YMMV.</font></tt><br><br><font size="3">On 6 November 2017 at 03:35, Carl <</font><a href="mailto:mutantllama@gmail.com" target="_blank"><font size="3" color="blue"><u>mutantllama@gmail.com</u></font></a><font size="3">>
wrote:</font><br></span><div><div class="h5"><font size="3">Hi Folk,</font><br><br><font size="3">Does anyone have much experience with the performance
of GPFS as it becomes close to full. In particular I am referring to split
data/meta data, where the data pool goes over 80% utilisation. </font><br><br><font size="3">How much degradation do you see above 80% usage, 90% usage?</font><br><br><font size="3">Cheers,</font><br><br><font size="3">Carl.</font><br><br><br></div></div><span class=""><font size="3"><br>______________________________<wbr>_________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org_&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=ql8RA3_vpc7iOubmRTKJDfyZ78LA2tNhXA5x2oIRzNk&e=" target="_blank"><font size="3" color="blue"><u>spectrumscale.org</u></font></a><font size="3" color="blue"><u><br></u></font><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=mnAoejImXp73wvcmpdLPXNi3JjPAmpAMDtzqIBnK7fs&e=" target="_blank"><font size="3" color="blue"><u>http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</u></font></a><font size="3"><br></font><br><font size="3"><br><br><br>-- </font><br></span><span class=""><font size="3" color="#4181c0"><b>Peter Smith</b> · Senior Systems Engineer<b><br>London</b> · New York · Los Angeles · Chicago · Montréal</font><font size="3"><br>T <a href="tel:+44%2020%207344%208000" value="+442073448000" target="_blank">+44 (0)20 7344 8000</a> · M </font><a href="tel:+44%20%280%297816%20123009" target="_blank"><font size="3" color="#0082bf">+44
(0)7816 123009</font></a><font size="3"> </font><font size="3" color="#0082bf"><i><br></i></font><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.google.co.uk_maps_place_19-2D23-2BWells-2BStreet-2C-2BLondon-2BW1T-2B3PQ&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=6acxGpabcJns6eGY7kvMNHkEl3PGz10xYjcsTVCjEiY&e=" target="_blank"><font size="3" color="#0082bf"><i>19-23
Wells Street, London W1T 3PQ</i></font></a><font size="3"> </font><font size="3" color="#0082bf"><br></font><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_framestore&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=Gmz8hKwZaaAM9ASBufpzpKG3_-nTjU8soMsH6j9ewAE&e=" target="_blank"><font size="3" color="#0082bf">Twitter</font></a><font size="3">· </font><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_framestore&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=fKTUmDeAcNoyVhMHbCLpBlJFtnLCJmToXgkHnQwup_E&e=" target="_blank"><font size="3" color="#0082bf">Facebook</font></a><font size="3">· </font><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__www.framestore.com_&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=8lv8uxKsqfE4NVwEEhC7KM_poV0Tzc2lF0pyfds3t0E&e=" target="_blank"><font size="3" color="#0082bf">framestore.com</font></a><font size="3" color="blue"><u><br></u></font><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.framestore.com_&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=Pfmf23Nhd4VeH3e8EPywrFcbegexNduC0Wi0CVlYjgI&e=" target="_blank"></a></span><span class=""><tt><font size="2">______________________________<wbr>_________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=lBgAfk6EsHjN156F5mngZVzk6argsIeu8gQlu9OWH_c&e=" target="_blank">spectrumscale.org</a><br></font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=mnAoejImXp73wvcmpdLPXNi3JjPAmpAMDtzqIBnK7fs&e=" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br></font></tt><br><br><br></span><p></p><p></p><p></p><br>______________________________<wbr>_________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=lBgAfk6EsHjN156F5mngZVzk6argsIeu8gQlu9OWH_c&e=" rel="noreferrer" target="_blank">spectrumscale.org</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=zatwEi1ZS1fdfJjSR0DnoHq8vd75vVJcEyrIbaU1Hyw&s=mnAoejImXp73wvcmpdLPXNi3JjPAmpAMDtzqIBnK7fs&e=" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br><br></blockquote></div><br></div></div></blockquote></div>Unless stated otherwise above:<BR>
IBM United Kingdom Limited - Registered in England and Wales with number 741598. <BR>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU<BR>
<BR>
</body></html>