<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:9pt" ><div dir="ltr" ><div>> we had left out "gpfs" from the<br>> vfs objects =<br>> line in smb.conf<br>><br>> so setting<br>> vfs objects = gpfs (etc)<br>> gpfs:hsm = yes<br>> gpfs:recalls = yes (not "no" as I had originally, and is implied by the manual)</div>
<div> </div>
<div>Thank you for the update. gpfs:recalls=yes is the default, allowing<br>recalls of files. If you set that to 'no', Samba will deny access to<br>"OFFLINE" files in GPFS through SMB.</div>
<div> </div>
<div>> and setting the offline flag on the file by migrating it, so that<br>> # mmlsattr -L filename.jpg<br>> ...<br>> Misc attributes: ARCHIVE OFFLINE<br>><br>> now Explorer on Windows 7 and 10 do not recall the file while viewing the folder with "Large icons"<br>><br>> and a standard icon with an X is displayed.<br>><br>> But after the file is then opened and recalled, the icon displays the thumbnail image and the OFFLINE flag is lost.</div>
<div> </div>
<div>Yes, that is working as intended. While the file is only in the<br>"external pool" (e.g. HSM tape), the OFFLINE flag is reported. Once<br>you read/write data, that triggers a recall to the disk pool and the<br>flag is cleared.</div>
<div> </div>
<div>> Also as you observed, Finder on MacOSX 10.13 ignores the file's offline flag,<br>><br>> so we still risk a recall storm caused by them.</div>
<div> </div>
<div>The question here would be how to handle the Mac clients. You could<br>configured two SMB shares on the same path: One with gpfs:recalls=yes<br>and tell the Windows users to access that share; the other one with<br>gpfs:recalls=no and tell the Mac users to use that share. That would<br>avoid the recall storms, but runs the risk of Mac users connecting to<br>the wrong share and avoiding this workaround...</div>
<div> </div>
<div>Regards,</div></div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div dir="ltr" ><br><span style="font-size:0.857em;" ><span style="font-family: Verdana,Geneva,sans-serif;" >Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ<br>christof.schmitt@us.ibm.com || +1-520-799-2469 (T/L: 321-2469)</span></span></div></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Cameron Dunn <Cameron.Dunn@bristol.ac.uk><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: "gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: Re: [gpfsug-discuss] preventing HSM tape recall storms<br>Date: Sat, Jul 7, 2018 2:30 PM<br> <br><!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" ><div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" ><p style="margin-top:0;margin-bottom:0" >Thanks Christof,</p>
<p style="margin-top:0;margin-bottom:0" > </p>
<p style="margin-top:0;margin-bottom:0" ><span style="font-size: 12pt;" >we had left out "gpfs" from the</span></p>
<div>vfs objects = </div>
<div>line in <span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;" >smb.conf</span></div>
<div> </div>
<div>so setting</div>
<div><span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;" >vfs objects = gpfs (etc)</span></div>
<div>gpfs:hsm = yes</div>
<div>gpfs:recalls = yes (not "no" as I had originally, and is implied by the manual)</div>
<div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" > </div>
<div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" >and setting the offline flag on the file by migrating it, so that</div>
<div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" ><div># mmlsattr -L filename.jpg</div>
<div>...</div>
<div> </div>
<div>Misc attributes: ARCHIVE OFFLINE</div></div>
<div dir="ltr" id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" > </div>
<p style="margin-top:0;margin-bottom:0" ><span>now Explorer on Windows 7 and 10 do not recall the file while viewing the folder with "Large icons"</span></p>
<p style="margin-top:0;margin-bottom:0" ><span>and a standard icon with an X is displayed.</span></p>
<p style="margin-top:0;margin-bottom:0" ><span>But after the file is then opened<span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;" > and<span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;" > recalled, </span>the icon displays the </span><span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;" >thumbnail image</span> and the OFFLINE flag is lost.</span></p>
<p style="margin-top:0;margin-bottom:0" > </p>
<p style="margin-top:0;margin-bottom:0" ><span style="font-size: 12pt;" >Also as you observed, Finder on MacOSX 10.13 ignores the file's offline flag,</span></p>
<p style="margin-top:0;margin-bottom:0" ><span style="font-size: 12pt;" >so we still risk a recall storm caused by them.</span></p>
<p style="margin-top:0;margin-bottom:0" > </p>
<p style="margin-top:0;margin-bottom:0" >All the best,</p>
<p style="margin-top:0;margin-bottom:0" >Cameron</p>
<p style="margin-top:0;margin-bottom:0" > </p></div>
<hr style="display:inline-block;width:98%" tabindex="-1" ><div dir="ltr" id="divRplyFwdMsg" ><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000" ><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Christof Schmitt <christof.schmitt@us.ibm.com><br><b>Sent:</b> 03 July 2018 20:37:08<br><b>To:</b> gpfsug-discuss@spectrumscale.org<br><b>Cc:</b> gpfsug-discuss@spectrumscale.org<br><b>Subject:</b> Re: [gpfsug-discuss] preventing HSM tape recall storms</font>
<div> </div></div>
<div><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif; font-size:9pt" ><div dir="ltr" ><div>> HSM over LTFS-EE runs the risk of a recall storm if files which have been migrated to tape<br>> are then shared by Samba to Macs and PCs.<br>> MacOS Finder and Windows Explorer will want to display all the thumbnail images of a<br>> folder's contents, which will recall lots of files from tape.</div>
<div> </div>
<div>SMB clients can query file information, including the OFFLINE<br>flag. With Spectrum Scale and the "gpfs" module loaded in Samba that<br>is mapped from the the OFFLINE flag that is visible in "mmlsattr<br>-L". In those systems, the SMB client can determine that a file is<br>offline.</div>
<div> </div>
<div>In our experience this is handled correctly in Windows Explorer; when<br>an "offline" file is encountered, no preview is generated from the<br>file data. The Finder on Mac clients does not seem to honor the<br>OFFLINE flag, thus the main problems are typically recall storms<br>caused by Mac clients.</div>
<div> </div>
<div>> According to the Samba documentation this is preventable by setting the following<br>> ----------------------------------------------<br>> <a href="https://www.samba.org/samba/docs/current/man-html/vfs_gpfs.8.html" target="_blank">https://www.samba.org/samba/docs/current/man-html/vfs_gpfs.8.html</a><br>><br>> gpfs:recalls = [ yes | no ]<br>> When this option is set to no, an attempt to open an offline file<br>> will be rejected with access denied.<br>> This helps preventing recall storms triggered by careless applications like Finder and Explorer.<br>><br>> yes(default) - Open files that are offline. This will recall the files from HSM.<br>> no - Reject access to offline files with access denied. This will prevent recalls of files from HSM.<br>> Using this setting also requires gpfs:hsm to be set to yes.<br>><br>> gpfs:hsm = [ yes | no ]<br>> Enable/Disable announcing if this FS has HSM enabled.<br>> no(default) - Do not announce HSM.<br>> yes - Announce HSM.<br>> --------------------------------------------------<br>><br>> However we could not get this to work.<br>><br>> On Centos7/Samba4.5, smb.conf contained<br>> gpfs:hsm = yes<br>> gpfs:recalls = no<br>> (also tried setting gpfs:offline = yes, though this is not documented)</div>
<div> </div>
<div>These options apply to the "gpfs" module in Samba. The Samba version<br>you are using needs to be built with GPFS support and the "gpfs"<br>module needs to be loaded through the "vfs objects" configuration. As<br>Centos7/Samba4.5 is mentioned, would guess that the CentOS provided<br>Samba version is used, which is probably not compiled with GPFS<br>support.</div>
<div> </div>
<div>From IBM we would recommend to use CES for protocol services, which<br>also provides Samba for SMB. The Samba provided through CES is<br>configured so that the gpfs:recalls option can be used:</div>
<div> </div>
<div><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmsmb.htm" target="_blank">https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmsmb.htm</a></div>
<div> </div>
<div>gpfs:recalls<br> If the value is set as yes files that have been migrated from disk<br> will be recalled on access. By default, this is enabled. If<br> recalls = no files will not be recalled on access and the client<br> will receive ACCESS_DENIED message.</div>
<div> </div>
<div>> We made a share containing image files that were then migrated to tape by LTFS-EE,<br>> to see if these flags were respected by OS X Finder or Windows Explorer.<br>><br>> Neither Mac OS X (using SMB3) or Windows 7 (using SMB2) respected the settings,<br>> so that when browsing the stubs in the share, the files were recalled from tape<br>> and the thumbnails displayed.<br>><br>> Has anyone seen these flags working as they are supposed to ?</div>
<div> </div>
<div>Yes, they are working, as we use them in our Samba build. Debugging<br>this would require looking at the Samba configuration and possibly<br>collecting a trace. If my above assumption was wrong and this problem<br>occurs with the CES Samba (gpfs.smb), please open a PMR for debugging<br>this issue. If this is not the CES Samba, please contact the provider<br>of the Samba package for additional support.</div>
<div> </div>
<div>Regards,</div>
<div> </div>
<div>Christof</div></div>
<div dir="ltr" ><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif; font-size:10.5pt" ><div dir="ltr" style="font-family:Arial; font-size:10.5pt" ><div dir="ltr" ><br><span style="font-size:0.857em" ><span style="font-family:Verdana,Geneva,sans-serif" >Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ<br>christof.schmitt@us.ibm.com || +1-520-799-2469 (T/L: 321-2469)</span></span></div></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Cameron Dunn <Cameron.Dunn@bristol.ac.uk><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: "gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: [gpfsug-discuss] preventing HSM tape recall storms<br>Date: Tue, Jul 3, 2018 6:22 AM<br>
<div dir="ltr" id="x_divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" ><p style="margin-top:0; margin-bottom:0" > </p>
<div>HSM over LTFS-EE runs the risk of a recall storm if files which have been migrated to tape </div>
<div>are then shared by Samba to Macs and PCs. </div>
<div>MacOS Finder and Windows Explorer will want<span style="font-size:12pt" > to display all the </span><span style="font-size:12pt" >thumbnail images of a </span></div>
<div><span style="font-size:12pt" >folder's contents, </span>which will recall lots of files from tape.</div>
<div> </div>
<div>According to the Samba documentation this is preventable by setting the following </div>
<div>----------------------------------------------</div>
<div>https://www.samba.org/samba/docs/current/man-html/vfs_gpfs.8.html</div>
<div> </div>
<div>gpfs:recalls = [ yes | no ]</div>
<div>When this option is set to no, an attempt to open an offline file </div>
<div>will be rejected with access denied. </div>
<div>This helps preventing recall storms triggered by careless applications like Finder and Explorer.</div>
<div> </div>
<div>yes(default) - Open files that are offline. This will recall the files from HSM.</div>
<div>no - Reject access to offline files with access denied. This will prevent recalls of files from HSM. </div>
<div>Using this setting also requires gpfs:hsm to be set to yes.</div>
<div> </div>
<div>gpfs:hsm = [ yes | no ]</div>
<div>Enable/Disable announcing if this FS has HSM enabled.</div>
<div>no(default) - Do not announce HSM.</div>
<div>yes - Announce HSM.</div>
<div>--------------------------------------------------</div>
<div> </div>
<div>However we could not get this to work.</div>
<div> </div>
<div>On Centos7/Samba4.5, smb.conf contained</div>
<div><div>gpfs:hsm = yes</div>
<div>gpfs:recalls = no</div>
<div><span style="font-size:12pt" >(also tried setting </span><span style="font-size:12pt" >gpfs:offline = yes, though this is not documented)</span></div>
<div> </div></div>
<div>We made a share containing image files that were then migrated to tape by LTFS-EE, </div>
<div>to see if these flags were respected by OS X Finder or Windows Explorer.</div>
<div> </div>
<div>Neither Mac OS X (using SMB3) or Windows 7 (using SMB2) respected the settings,</div>
<div>so that when browsing the stubs in the share, the files were recalled from tape</div>
<div>and the thumbnails displayed.</div>
<div> </div>
<div>Has anyone seen these flags working as they are supposed to ?</div>
<p> </p>
<p style="margin-top:0; margin-bottom:0" >Many thanks for any ideas,</p>
<p style="margin-top:0; margin-bottom:0" >Cameron</p>
<p style="margin-top:0; margin-bottom:0" > </p>
<div id="x_Signature" ><div dir="ltr" id="x_divtagdefaultwrapper" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols" ><div dir="ltr" ><div>Cameron Dunn<br>Advanced Computing Systems Administrator</div>Advanced Computing Research Centre</div>
<div dir="ltr" >University of Bristol</div></div></div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></div></blockquote>
<div dir="ltr" > </div></div></div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></div></blockquote>
<div dir="ltr" > </div></div><BR>