<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p><span style="font-size: 12pt;">Since I solved this old issue a long time ago</span><span style="font-size: 12pt;">, I'd thought I'd come back and report the solution in case someone else encounters similar problems in the future.</span></p>
<p><br>
</p>
<p><br>
</p>
<p>Original problem reported by users:</p>
<p>Copying files between folders on NFS exports from a CES server gave random timestamps on the files. Also, apart from the initial reported problem, there where issues where users sometimes couldn't change or delete files that they where owners of.</p>
<p><br>
</p>
<p><span style="font-size: 12pt;">Background</span><span style="font-size: 12pt;">:</span><br>
</p>
<p>We have a Active Directory with RFC2307 posix attributes populated, and use the built in Winbind-based AD authentication with RFC2307 ID mapping of our Spectrum Scale CES protocol servers.</p>
<p>All our Linux clients and servers are also AD integrated, using Nslcd and nss-pam-ldapd.</p>
<p><br>
</p>
<p><span style="font-size: 12pt;">Trigger:</span><br>
</p>
<p>If a user was part of a AD group with a mixed case name, and this group gave access to a folder, and the NFS mount was done using NFSv4, the behavior in my original post occurred when copying or changing files in that folder.</p>
<p><br>
</p>
<p>Cause:</p>
<p></p>
<p style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;"><span style="font-size: 12pt;">Active Directory handle LDAP-requests case insensitive, but results are returned with case retained</span><span style="font-size: 12pt;">.</span><br>
</p>
<p style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">Winbind and SSSD-AD converts groups and usernames to lower case. Nslcd retains case.</p>
<p style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">We run NFS with managed GIDs. Managed GIDs in NFSv3 seems to be handled case insensitive, or to ignore the actual group name after it has resolved the GID-number of the group, while
NFSv4 seems to handle group names case sensitive and check the actual group name for certain operations even if the GID-number matches.</p>
<p style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">Don't fully understand the mechanism behind why certain file operations would work but others not, but in essence a user would be part of a group called "UserGroup" with GID-number
1234 in AD and on the client, but would be part of a group called "usergroup" with GID-number 1234 on the CES server.</p>
<p></p>
<p>Any operation that's authorized on the GID-number, or a case insensitive lookup of the group name, would work. Any operation authorized by a case sensitive group lookup would fail.</p>
<p><br>
</p>
<p>Three different workarounds where found to work:</p>
<p>1. Rename groups and users to lower case in AD</p>
<p>2. Change from Nslcd to either SSSD or Winbind on the clients</p>
<p>3. Change from NFSv4 to NFSv3 when mounting NFS</p>
<p><br>
</p>
<p>Remember to clear ID-mapping caches.</p>
<p><br>
</p>
<p>Regards,</p>
<p>Andreas</p>
<div id="Signature">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px"><br>
</div>
<div style="font-family:Tahoma; font-size:13px"><span class="HOEnZb adL"><font color="#888888">___________________________________<br>
<a href="https://www.maxlab.lu.se" target="_blank" id="LPNoLP"><img width="130" style="border-style: none; user-select: none;" src="https://mail.google.com/mail/u/0/?ui=2&ik=b0a6f02971&view=att&th=14618fab2daf0e10&attid=0.1.1&disp=emb&zw&atsh=1"></a><br>
<br>
<font size="3"><b>Andreas Mattsson</b><br>
<font size="2">System Engineer<br>
<br>
<span class="il">MAX</span> IV Laboratory<br>
Lund University<br>
Tel: <a href="tel:%2B46-706-649544" id="LPNoLP"><font color="3366FF">+46-706-649544</font></a><br>
E-mail: <a href="mailto:andreas.mattsson@maxlab.lu.se" id="LPNoLP"><font color="3366FF">andreas.mattsson@maxlab.lu.se</font></a><br>
<a href="mailto:daniel.liikamaa@maxlab.lu.se" target="_blank" id="LPNoLP"><span class="il"></span></a></font></font></font></span><br>
</div>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>Från:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> för Stephen Ulmer <ulmer@ulmer.org><br>
<b>Skickat:</b> den 3 februari 2017 14:35:21<br>
<b>Till:</b> gpfsug main discussion list<br>
<b>Ämne:</b> Re: [gpfsug-discuss] Strange timestamp behaviour on NFS via CES</font>
<div> </div>
</div>
<div>Does the cp actually complete? As in, does it copy all of the blocks? What’s the exit code?
<div class=""><br class="">
</div>
<div class="">A cp’d file should have “new” metadata. That is, it should have it’s own dates, owners, etc. (not necessarily copied from the source file).</div>
<div class=""><br class="">
</div>
<div class="">I ran ‘strace cp foo1 foo2’, and it was pretty instructive, maybe that would get you more info. On CentOS strace is in it’s own package, YMMV.</div>
<div class=""><br class="">
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
-- <br class="">
Stephen<br class="">
<br class="">
<br class="">
</div>
</div>
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 3, 2017, at 8:19 AM, Andreas Mattsson <<a href="mailto:andreas.mattsson@maxiv.lu.se" class="">andreas.mattsson@maxiv.lu.se</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">That works.<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">’touch test100’<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Feb 3 14:16 test100<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">‘cp test100 test101’<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Feb 3 14:16 test100<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Apr 21 2027 test101<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">‘touch –r test100 test101’<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Feb 3 14:16 test100<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Feb 3 14:16 test101<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">/Andreas<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" class="">That’s a cool one. :)<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" class=""><o:p class=""> </o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
What if you use the "random date" file as a time reference to touch another file (like, 'touch -r file02 file03’)?<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<p class="MsoNormal" style="margin: 0cm 0cm 12pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="" class="">-- <br class="">
Stephen<br class="">
<br class="">
<o:p class=""></o:p></span></p>
</div>
</div>
</div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
On Feb 3, 2017, at 7:46 AM, Andreas Mattsson <<a href="mailto:andreas.mattsson@maxiv.lu.se" style="color: purple; text-decoration: underline;" class="">andreas.mattsson@maxiv.lu.se</a>> wrote:<o:p class=""></o:p></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-GB" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">I’m having some really strange timestamp behaviour when doing file operations on NFS mounts shared via CES on spectrum scale 4.2.1.1</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">The NFS clients are up to date Centos and Debian machines.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">All Scale servers and NFS clients have correct date and time via NTP.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Creating a file, for instance ‘touch file00’, gives correct timestamp.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Moving the file, ‘mv file00 file01’, gives correct timestamp</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Copying the file, ‘cp file01 file02’, gives a random timestamp anywhere in time, for instance Oct 12 2095 or Feb 29 1976 or something similar.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">This is only via NFS. Copying the file via a native gpfs-mount or via SMB gives a correct timestamp.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Doing the same operation over NFS to other NFS-servers works correct, it is only when operating on the NFS-share from the Spectrum Scale CES the issue occurs.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Have anyone seen this before?</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Regards,</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Andreas Mattsson</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" class="">_____________________________________________</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: 'Lucida Grande';" class=""><image001.png></span><span lang="EN-US" style="font-size: 11pt; font-family: 'Lucida Grande';" class=""><br class="">
</span><b class=""><span lang="EN-US" style="font-size: 10pt; font-family: Calibri, sans-serif;" class=""> <br class="">
Andreas Mattsson<br class="">
</span></b><span lang="EN-US" style="font-size: 8pt; font-family: Calibri, sans-serif;" class="">Systems Engineer</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<p class="MsoNormal" style="margin: 0cm 0cm 12pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span lang="EN-US" style="font-size: 8pt; font-family: Calibri, sans-serif;" class="">MAX IV Laboratory<br class="">
Lund University<br class="">
</span><span lang="EN-US" style="font-size: 8.5pt;" class="">P.O. Box 118, SE-221 00 Lund, Sweden<br class="">
Visiting address: Fotongatan 2, 225 94 Lund<br class="">
</span><span lang="EN-US" style="font-size: 8pt; font-family: Calibri, sans-serif;" class="">Mobile: +46 706 64 95 44<br class="">
</span><span style="font-size: 8pt; font-family: Calibri, sans-serif;" class=""><a href="mailto:andreas.mattsson@maxiv.se" style="color: purple; text-decoration: underline;" class=""><span lang="EN-US" class="">andreas.mattsson@maxiv.se</span></a></span><span lang="EN-US" style="font-size: 8pt; font-family: Calibri, sans-serif;" class=""><br class="">
</span><span style="font-size: 8pt; font-family: Calibri, sans-serif;" class=""><a href="http://www.maxiv.se/" style="color: purple; text-decoration: underline;" class=""><span lang="EN-US" class="">www.maxiv.se</span></a></span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></p>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at<span class="apple-converted-space"> </span></span><a href="http://spectrumscale.org/" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class="">spectrumscale.org</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">
</span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></a><o:p class=""></o:p></div>
</div>
</blockquote>
</div>
<div style="margin: 0cm 0cm 0.0001pt 65.2pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
</div>
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">gpfsug-discuss
mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">gpfsug-discuss
at<span class="Apple-converted-space"> </span></span><a href="http://spectrumscale.org/" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">spectrumscale.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</body>
</html>