<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >The general strategy is this: when you open a file and start writing, we'll allocate full blocks. When you close the file we look at file size and replace the last block with a fragment just big enough to hold the data at the end of the file ("shrink-to-fit").</div>
<div dir="ltr" > </div>
<div dir="ltr" >If you open the file again later and append new data that doesn't fit in the existing fragment, we'll first allocate another full block ("fragment expansion"), then shrink again when you close the file again.</div>
<div dir="ltr" > </div>
<div dir="ltr" >Note that we don't shrink or expand "in place", but rather first allocate a whole new fragment or block somewhere else, then free the old block or fragment that's no longer needed. That's done to minimize space fragmentation, because shrink-to-fit will be able to use left over free space in another block that holds fragments from other files already, rather than cutting up the full block that was originally allocated for the last block of the file.</div>
<div dir="ltr" > </div>
<div dir="ltr" >In your example, where you only append once every 24 hours, that means your fragment will keep moving around once a day, as the fragment grows, until it has become a full block. Normally, if writes/appends occur close enough in time, dirty data in the last block tends to stay in the buffer pool long enough that the final shrink-to-fit happens before dirty data is flushed to disk, so data is only written once, to its final location.</div>
<div dir="ltr" > </div>
<div dir="ltr" > -- Frank</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Daniel Kidger" <daniel.kidger@uk.ibm.com><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: "gpfsug main discussion list" <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: Re: [gpfsug-discuss] file layout API + file fragmentation<br>Date: Mon, Nov 6, 2017 11:01 AM<br> <br><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->Frank,
<div> </div>
<div>For clarity in the understanding the underlying mechanism in GPFS, could you describe what happens in the case say of a particular file that is appended to every 24 hours? <br>ie. as that file gets to 7MB, it then writes to a new sub-block (1/32 of the next 1MB block). I guess that sub block could be 10th in a a block that already has 9 used. Later on, the file grows to need an 11th subblock and so on. So at what point does this growing file at 8MB occupy all 32 sunblocks of 8 full blocks?</div>
<div>
<div id="AppleMailSignature" ><div style="margin-bottom: 0.0001pt; line-height: normal;" ><span style="background-color: rgba(255, 255, 255, 0);" >Daniel</span></div>
<div style="margin-bottom: 0.0001pt; line-height: normal;" ><span style="background-color: rgba(255, 255, 255, 0);" ><img alt="/spectrum_storage-banne" height="auto" s_org_src="http://ausgsa.ibm.com/projects/t/tivoli_visual_design/public/2015/Spectrum-Storage/Email-signatures/Storage/spectrum_storage-banner.png" style="width: 601px; height: 5px;" width="432.5" ></span></div>
<div style="margin-bottom: 0.0001pt; line-height: normal;" ><br><span style="background-color: rgba(255, 255, 255, 0);" > </span></div>
<table border="0" cellpadding="0" cellspacing="0" style="font-family: sans-serif;" > <tbody> <tr style="word-wrap: break-word; max-width: 447.5px;" > <td style="word-wrap: break-word; max-width: 447.5px; width: 201px; padding: 0cm;" > <div style="margin-bottom: 0.0001pt; line-height: normal;" ><img alt="Spectrum Scale Logo" s_org_src="http://ausgsa.ibm.com/projects/t/tivoli_visual_design/public/2015/Spectrum-Storage/Email-signatures/Storage/spectrum_scale-logo.png" style="width: 75px; height: 120px; float: left;" ></div>
<div style="margin-bottom: 0.0001pt; line-height: normal;" ><font face=".SFUIDisplay" ><span style="background-color: rgba(255, 255, 255, 0);" > </span></font></div> </td> <td style="word-wrap: break-word; max-width: 447.5px; width: 21px; padding: 0cm;" ><font face=".SFUIDisplay" ><span style="background-color: rgba(255, 255, 255, 0);" > </span></font></td> <td style="word-wrap: break-word; max-width: 447.5px; width: 202px; padding: 0cm;" > <div style="margin-bottom: 0.0001pt; line-height: normal;" ><font face=".SFUIDisplay" ><span style="background-color: rgba(255, 255, 255, 0);" ><strong>Dr Daniel Kidger</strong> <br> IBM Technical Sales Specialist<br> Software Defined Solution Sales<br> <br> <a dir="ltr" href="tel:+%2044-7818%20522%20266" target="_blank" x-apple-data-detectors="true" x-apple-data-detectors-result="1" x-apple-data-detectors-type="telephone" >+</a> <a dir="ltr" href="tel:+%2044-7818%20522%20266" target="_blank" x-apple-data-detectors="true" x-apple-data-detectors-result="1" x-apple-data-detectors-type="telephone" >44-(0)7818 522 266</a> <br> <a dir="ltr" href="mailto:daniel.kidger@uk.ibm.com" target="_blank" x-apple-data-detectors="true" x-apple-data-detectors-result="2" x-apple-data-detectors-type="link" >daniel.kidger@uk.ibm.com</a></span></font></div> </td> </tr> </tbody></table></div>
<div><br>On 6 Nov 2017, at 00:57, Frank Schmuck <<a href="mailto:fschmuck@us.ibm.com" target="_blank" >fschmuck@us.ibm.com</a>> wrote:<br> </div>
<blockquote type="cite" ><div><div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >In GPFS blocks within a file are never fragmented. For example, if you have a file of size 7.3 MB and your file system block size is 1MB, then this file will be made up of 7 full blocks and one fragment of size 320k (10 subblocks). Each of the 7 full blocks will be contiguous on a singe diks (LUN) behind a single NSD server. The fragment that makes up the last part of the file will also be contiguous on a single disk, just shorter than a full block.</div>
<div dir="ltr" > </div>
<div dir="ltr" >Frank Schmuck<br>IBM Almaden Research Center<br> </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Aaron Knister <<a href="mailto:aaron.s.knister@nasa.gov" target="_blank" >aaron.s.knister@nasa.gov</a>><br>Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank" >gpfsug-discuss-bounces@spectrumscale.org</a><br>To: <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>><br>Cc:<br>Subject: Re: [gpfsug-discuss] file layout API + file fragmentation<br>Date: Sun, Nov 5, 2017 3:39 PM<br>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >Thanks Marc, that helps. I can't easily use tsdbfs for what I'm working<br>on since it needs to be run as unprivileged users.<br><br>Perhaps I'm not asking the right question. I'm wondering how the file<br>layout api behaves if a given "block"-aligned offset in a file is made<br>up of sub-blocks/fragments that are not all on the same NSD. The<br>assumption based on how I've seen the API used so far is that all<br>sub-blocks within a block at a given offset within a file are all on the<br>same NSD.<br><br>-Aaron<br><br>On 11/5/17 6:01 PM, Marc A Kaplan wrote:<br>> I googled GPFS_FCNTL_GET_DATABLKDISKIDX<br>><br>> and found this discussion:<br>><br>> Â <a href="https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50" target="_blank" >https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50</a><br>><br>> In general, GPFS files ARE deliberately "fragmented" but we don't say<br>> that - we say they are "striped" over many disks -- and that is<br>> generally a good thing for parallel performance.<br>><br>> Also, in GPFS, if the last would-be block of a file is less than a<br>> block, then it is stored in a "fragment" of a block. Â <br>> So you see we use "fragment" to mean something different than other file<br>> systems you may know.<br>><br>> --marc<br>><br>><br>><br>> From: Â Â Â Â Aaron Knister <<a href="mailto:aaron.s.knister@nasa.gov" target="_blank" >aaron.s.knister@nasa.gov</a>><br>> To: Â Â Â Â gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>><br>> Date: Â Â Â Â 11/04/2017 12:22 PM<br>> Subject: Â Â Â Â [gpfsug-discuss] file layout API + file fragmentation<br>> Sent by: Â Â Â Â <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank" >gpfsug-discuss-bounces@spectrumscale.org</a><br>> ------------------------------------------------------------------------<br>><br>><br>><br>> I've got a question about the file layout API and how it reacts in the<br>> case of fragmented files.<br>><br>> I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure and have some code<br>> based on tsGetDataBlk.C. I'm basing the block size based off of what's<br>> returned by filemapOut.blockSize but that only seems to return a value ><br>> 0 when filemapIn.startOffset is 0.<br>><br>> In a case where a file were to be made up of a significant number of<br>> non-contiguous fragments (which... would be awful in of itself) how<br>> would this be reported by the file layout API? Does the interface<br>> technically just report the disk location information of the first block<br>> of the $blockSize range and assume that it's contiguous?<br>><br>> Thanks!<br>><br>> -Aaron<br>><br>> --<br>> Aaron Knister<br>> NASA Center for Climate Simulation (Code 606.2)<br>> Goddard Space Flight Center<br>> (301) 286-2776<br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=Irfx9Rw_4XM_6qhDmqs56NYXRVMs9hXljWesk3gY09Q&e=" target="_blank" >spectrumscale.org</a><br>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e=</a><br>><br>><br>><br>><br>><br>><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=Irfx9Rw_4XM_6qhDmqs56NYXRVMs9hXljWesk3gY09Q&e=" target="_blank" >spectrumscale.org</a><br>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=</a><br>><br><br>--<br>Aaron Knister<br>NASA Center for Climate Simulation (Code 606.2)<br>Goddard Space Flight Center<br>(301) 286-2776<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=Irfx9Rw_4XM_6qhDmqs56NYXRVMs9hXljWesk3gY09Q&e=" target="_blank" >spectrumscale.org</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=</a></font><br> </div></blockquote>
<div dir="ltr" > </div></div>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=Irfx9Rw_4XM_6qhDmqs56NYXRVMs9hXljWesk3gY09Q&e=" target="_blank" >spectrumscale.org</a><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=WH1GLDCza1Rvd9bzdVYoz2Pdzs7l90XNnhUb40FYCqQ&s=LOkUY79m5Ow2FeKqfCqc31cfXZVmqYlvBuQRPirGOFU&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=WH1GLDCza1Rvd9bzdVYoz2Pdzs7l90XNnhUb40FYCqQ&s=LOkUY79m5Ow2FeKqfCqc31cfXZVmqYlvBuQRPirGOFU&e=</a></font></div></div></blockquote></div>Unless stated otherwise above:<br>IBM United Kingdom Limited - Registered in England and Wales with number 741598.<br>Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU<br>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=JpABFLjCrCbNPSQi7CjZqiDVla-2Z0rvm980yvSRIj4&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=so8gVkqBwkLKe8hIYlogJ_Dror0uoEEfh4GWAd68X6U&s=JpABFLjCrCbNPSQi7CjZqiDVla-2Z0rvm980yvSRIj4&e=</a></font></div></blockquote>
<div dir="ltr" > </div></div></div><BR>