<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><font face="monospace">Hi Christof,</font></p>
<p><font face="monospace">thanks! So the preferred way should be not
to have Filesystem_Id. If I have understood correctly, this is
anyhow set up by default in CES to 666.666, so the suggested
procedure during the CES setup should be to manually modify
gpfs.ganesha.exports.conf and remove this parameter from all the
exports, is that correct? Is there an easier way, or is there a
plan to remove the 666.666 default value?</font></p>
<p><font face="monospace">In our case, we do rely on two separated
CES clusters (one in prod and one in stand by, so that we can
perform upgrades with no downtime by migrating the IPs from one
to the other), so it might be safer to explicilty set
Filesystem_Id, to ensure consistency among the cluster - would
that make sense?</font></p>
<p><font face="monospace">Thanks again!</font></p>
<p><font face="monospace">Regards</font></p>
<p><font face="monospace">leo</font><br>
</p>
<pre class="moz-signature" cols="72">Paul Scherrer Institut
Dr. Leonardo Sala
Group Leader Data Analysis and Research Infrastructure
Deputy Department Head a.i Science IT Infrastructure and Services department
Science IT Infrastructure and Services department (AWI)
WHGA/036
Forschungstrasse 111
5232 Villigen PSI
Switzerland
Phone: +41 56 310 3369
<a class="moz-txt-link-abbreviated" href="mailto:leonardo.sala@psi.ch">leonardo.sala@psi.ch</a>
<a class="moz-txt-link-abbreviated" href="http://www.psi.ch">www.psi.ch</a></pre>
<div class="moz-cite-prefix">On 6/28/23 15:22, Christof Schmitt
wrote:<br>
</div>
<blockquote type="cite" cite="mid:46f2c6856498222769ed523483a14c04ffb6a858.camel@us.ibm.com">
<div>After another discussion it turns out that this parameter is
not required. While my previous comment is correct, that there
is the need to have unique handles across file systems, GPFS
already provides that information and Ganesha handles that
correctly. So there is no need to set the parameter in the
Ganesha config.</div>
<div>Regards,</div>
<div><br>
</div>
<div>Christof</div>
<div><br>
</div>
<div>On Wed, 2023-06-28 at 14:33 +0200, Leonardo Sala wrote:</div>
<blockquote type="cite">
<div>
Hi Christof, thanks a lot! In our case we are exporting
multiple filesets from 2 filesystems, I guess we should fix
unique Fileset_IDs for each fileset? What would happen in case
we just remove the Fileset_id parameter, as suggested by the
</div>
<div>
ZjQcmQRYFpfptBannerStart</div>
<div dir="ltr" id="pfptBanners6ux0e7" lang="en">
<div id="pfptBanners6ux0e7">
<div id="pfptBanners6ux0e7">
This Message Is From an External Sender </div>
<div id="pfptBanners6ux0e7">
This message came from outside your organization. </div>
</div>
<div id="pfptBanners6ux0e7">
<a id="pfptBanners6ux0e7" href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/PjiDSg!2k-hAt5_AJM6OsYbOMsPTZmPil8BIAHadeloM-6vntuXy4WCtpq_71hHj07-QEB3YfmuMTIe9Ezg-qdLzL_6DvzGmA12g21EJgQ$" moz-do-not-send="true">
<div class="pfptPrimaryButtons6ux0e7">Report Suspicious </div>
</a></div>
<div>
</div>
</div>
<div>
ZjQcmQRYFpfptBannerEnd</div>
<p>Hi Christof,</p>
<p>thanks a lot! In our case we are exporting multiple filesets
from 2 filesystems, I guess we should fix unique Fileset_IDs
for each fileset? What would happen in case we just remove the
Fileset_id parameter, as suggested by the ganesha docs?</p>
<p>Regards</p>
<p>leo<br>
</p>
<pre>Paul Scherrer Institut</pre>
<pre>Dr. Leonardo Sala</pre>
<pre>Group Leader Data Analysis and Research Infrastructure</pre>
<pre>Deputy Department Head a.i Science IT Infrastructure and Services department</pre>
<pre>Science IT Infrastructure and Services department (AWI)</pre>
<pre>WHGA/036</pre>
<pre>Forschungstrasse 111</pre>
<pre>5232 Villigen PSI</pre>
<pre>Switzerland</pre>
<pre>
</pre>
<pre>Phone: +41 56 310 3369</pre>
<a class="moz-txt-link-abbreviated" href="mailto:leonardo.sala@psi.ch" moz-do-not-send="true">
<pre>leonardo.sala@psi.ch</pre>
</a>
<pre>
</pre>
<a class="moz-txt-link-abbreviated" href="http://www.psi.ch" moz-do-not-send="true">
<pre>www.psi.ch</pre>
</a>
<div class="moz-cite-prefix">On 6/28/23 14:22, Christof Schmitt
wrote:<br>
</div>
<blockquote type="cite" cite="mid:558222eeef7cd5169214368c9a04af813509b086.camel@us.ibm.com">
<div>The "FileSystem_Id" is a unique identifier for the file
system. The technical background is that Ganesha asks the
file system for a file handle, but that is only unique
within the file system. If there are NFS exports on
different file systems, there needs to be a way to make the
file handles unique across multiple file systems. So if
there are NFS exports on different file systems, this
parameter should be set with a unique value for each file
system. If there is only one file system with NFS exports,
then this should not be necessary.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Christof</div>
<div><br>
</div>
<div>On Wed, 2023-06-28 at 08:53 +0200, Leonardo Sala wrote:</div>
<blockquote type="cite">
<div>Hi Ed, thanks! In our case we do have unique export
ids, but the same fsid, and this seems to create issues.
Also, reading Ganesha docs, I can see [*]: FileSystem_ID
EXPORT Option There is an EXPORT config option,
FileSystem_ID. This really
</div>
<div>ZjQcmQRYFpfptBannerStart</div>
<div dir="ltr" id="pfptBannerls5jz08" lang="en">
<div id="pfptBannerls5jz08">
<div id="pfptBannerls5jz08">This Message Is From an
External Sender </div>
<div id="pfptBannerls5jz08">This message came from
outside your organization. </div>
</div>
<div id="pfptBannerls5jz08"><a id="pfptBannerls5jz08" href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/PjiDSg!2k-ror9_yrma2gYRGMtvzfBw-f7-KrLp6ZZNwHChWJfOvNhVe0t6WfmouxHnuoc3gtMvXaxWV3AbFrgYAQT1JIEKoq1hoJ8cr1c$" moz-do-not-send="true">
<div class="pfptPrimaryButtonls5jz08">Report Suspicious
</div>
</a></div>
<div> </div>
</div>
<div>ZjQcmQRYFpfptBannerEnd</div>
<p>Hi Ed,</p>
<p>thanks! In our case we do have unique export ids, but the
same fsid, and this seems to create issues. Also, reading
Ganesha docs, I can see [*]:</p>
<h3><span>FileSystem_ID EXPORT Option</span></h3>
<p>There is an EXPORT config option, FileSystem_ID. This
really should not be used, all it does it designate an
fsid to be used with the attributes of all objects in the
export. It will be folded to fit into NFSv3. Because it
applies to the entire export, it prevents exporting
multiple file systems since there will likely be issues
with collision of inode numbers on the client.</p>
<p>so before touching the defaults in GPFS CES configuration
I would like some guidance or experiences from this mlist
:)</p>
<p>cheers</p>
<p>leo<br>
</p>
<p>[*] <a class="moz-txt-link-freetext" href="https://github.com/nfs-ganesha/nfs-ganesha/wiki/File-Systems#FileSystem_ID_EXPORT_Option" moz-do-not-send="true">https://github.com/nfs-ganesha/nfs-ganesha/wiki/File-Systems#FileSystem_ID_EXPORT_Option</a><br>
</p>
<p><br>
</p>
<pre>Paul Scherrer Institut</pre>
<pre>Dr. Leonardo Sala</pre>
<pre>Group Leader Data Analysis and Research Infrastructure</pre>
<pre>Deputy Department Head a.i Science IT Infrastructure and Services department</pre>
<pre>Science IT Infrastructure and Services department (AWI)</pre>
<pre>WHGA/036</pre>
<pre>Forschungstrasse 111</pre>
<pre>5232 Villigen PSI</pre>
<pre>Switzerland</pre>
<pre>Phone: +41 56 310 3369</pre>
<a class="moz-txt-link-abbreviated" href="mailto:leonardo.sala@psi.ch" moz-do-not-send="true">
<pre>leonardo.sala@psi.ch</pre>
</a><a class="moz-txt-link-abbreviated" href="http://www.psi.ch" moz-do-not-send="true">
<pre>www.psi.ch</pre>
</a>
<div class="moz-cite-prefix">On 6/27/23 19:41, Wahl, Edward
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CH0PR01MB6825595F142F5D9C9F269324A827A@CH0PR01MB6825.prod.exchangelabs.com">
<meta name="Generator" content="Microsoft Word 15
(filtered medium)">
<div class="WordSection1">
<p class="MsoNormal"><span>I vaguely recall seeing this
and testing it. My notes to myself say: ‘As long as
the export_id is unique, you are fine.’ See the
manuals, ganesha loves Camel Case so it’s more than
likely actually “Export_Id” or some such.
</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>Ed Wahl</span></p>
<p class="MsoNormal"><span>Ohio Supercomputer Center</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<p class="MsoNormal"><b><span>From:</span></b><span>
gpfsug-discuss <a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss-bounces@gpfsug.org" moz-do-not-send="true"><gpfsug-discuss-bounces@gpfsug.org></a>
<b>On Behalf Of </b>Leonardo Sala<br>
<b>Sent:</b> Tuesday, June 27, 2023 10:18 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated
moz-txt-link-freetext" href="mailto:gpfsug-discuss@spectrumscale.org" moz-do-not-send="true">gpfsug-discuss@spectrumscale.org</a><br>
<b>Subject:</b> [gpfsug-discuss] CES, Ganesha, and
Filesystem_id</span></p>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal"><span>Hallo, we are checking our
current CES configuration, and we noticed that by
default GPFS puts always Filesystem_Id=666. 666
[*], no matter which Export_Id value the export
has. To my understanding (which is poor!), this
means that all clients </span></p>
</div>
<div>
<p class="MsoNormal"><span></span></p>
</div>
<p><span>Hallo,</span></p>
<p><span>we are checking our current CES configuration,
and we noticed that by default GPFS puts always
Filesystem_Id=666.666 [*], no matter which Export_Id
value the export has. To my understanding (which is
poor!), this means that all clients will see all our
exports (~20) with the same device number, creating
various possible issues (e.g. file state handles).
Questions:</span></p>
<p><span>* is there a reason for such default value? If
we change it, are there unpleasant effects we could
see?</span></p>
<p><span>* what would be a reasonable value? Looking
around I saw that Filesystem_Id =
Export_Id.Export_Id is quite common, with the
possible issue of using the forbidden 152.152 [**]</span></p>
<p><span>* what happens if we actually remove the
Filesytem_Id parameter from
gpfs.ganesha.exports.conf?</span></p>
<p><span>* is there a way to modify Filesystem_Id in
gpfs.ganesha.exports.conf without editing the file,
eg using mmnfs commands (seems not, but I might be
mistaken)?</span></p>
<p><span>Thanks a lot!</span></p>
<p><span>cheers</span></p>
<p><span>leo</span></p>
<p> </p>
<p><span>[*] <a href="https://www.ibm.com/docs/en/storage-scale/5.0.4?topic=exports-making-bulk-changes-nfs" class="moz-txt-link-freetext" moz-do-not-send="true">https://www.ibm.com/docs/en/storage-scale/5.0.4?topic=exports-making-bulk-changes-nfs</a></span></p>
<p><span>[**] <a href="https://github.com/nfs-ganesha/nfs-ganesha/issues/615" class="moz-txt-link-freetext" moz-do-not-send="true">https://github.com/nfs-ganesha/nfs-ganesha/issues/615</a></span></p>
<pre>-- </pre>
<pre>Paul Scherrer Institut</pre>
<pre>Dr. Leonardo Sala</pre>
<pre>Group Leader Data Analysis and Research Infrastructure</pre>
<pre>Deputy Department Head a.i Science IT Infrastructure and Services department</pre>
<pre>Science IT Infrastructure and Services department (AWI)</pre>
<pre>WHGA/036</pre>
<pre>Forschungstrasse 111</pre>
<pre>5232 Villigen PSI</pre>
<pre>Switzerland</pre>
<pre> </pre>
<pre>Phone: +41 56 310 3369</pre>
<a href="mailto:leonardo.sala@psi.ch" class="moz-txt-link-freetext" moz-do-not-send="true">
<pre>leonardo.sala@psi.ch</pre>
</a><a href="http://www.psi.ch" moz-do-not-send="true">
<pre>www.psi.ch</pre>
</a></div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre>_______________________________________________</pre>
<pre>gpfsug-discuss mailing list</pre>
<pre>gpfsug-discuss at gpfsug.org</pre>
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org" moz-do-not-send="true">
<pre>http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</pre>
</a></blockquote>
<pre>_______________________________________________</pre>
<pre>gpfsug-discuss mailing list</pre>
<pre>gpfsug-discuss at gpfsug.org</pre>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org" moz-do-not-send="true">
<pre>http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</pre>
</a>
<pre> </pre>
</blockquote>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre>_______________________________________________</pre>
<pre>gpfsug-discuss mailing list</pre>
<pre>gpfsug-discuss at gpfsug.org</pre>
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org" moz-do-not-send="true">
<pre>http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</pre>
</a>
<pre>
</pre>
</blockquote>
</blockquote>
</blockquote>
</body>
</html>