<font size=2 face="Arial">Jez, </font>
<br>
<br><font size=2 face="Arial">the other replies to my email where kind
of unexpected as i was indeed intending to help, i will continue to provide
details and try to answer serious questions and comments. </font>
<br>
<br><font size=2 face="Arial">also as you point out there is no secret
master plan to spread misinformation, the topic is simply complicated and
there are reasons that are hard to explain why there is no full official
support outside the current shipping IBM products and i will not dive into
discussions around it. </font>
<br>
<br><font size=2 face="Arial">a few words of clarification, on the controversial
point, compile ctdb yes/no. </font>
<br><font size=2 face="Arial">while its technically correct, that you don't
have to recompile ctdb and it has no code that links/depends in any form
on GPFS (samba does), you still have to compile one if the one shipped
with your distro is out of date and while i didn't explicitly wrote it
this way, this is what i intended to say. </font>
<br>
<br><font size=2 face="Arial">the version RH ships is very old and since
has gotten many fixes which i highly recommend people to use unless the
distros will pickup more recent version like RHEL7 will do. i probably
wouldn't recommend a daily git build, but rather the packaged versions
that are hosted on ftp.samba.org like : </font><a href=https://ftp.samba.org/pub/ctdb/packages/redhat/RHEL6/><font size=2 color=blue face="Arial"><u>https://ftp.samba.org/pub/ctdb/packages/redhat/RHEL6/</u></font></a><font size=2 face="Arial">
</font>
<br>
<br><font size=2 face="Arial">so the proposed order of things would be
to install gpfs, pull the src package of ctdb, compile and install ctdb
and the devel packages, then pull a recent samba src package , install
all the dependencies and build samba on this same host with gpfs and ctdb
packages already installed. the resulting rpm's should contain the proper
code to continue.</font>
<br>
<br><font size=2 face="Arial">after you got your versions compiled, installed
and the basics of ctdb running, you should use the following smb.conf as
a starting point : </font>
<br>
<br><font size=2 face="Arial">[global]</font>
<br><font size=2 face="Arial"> netbios
name = cluster</font>
<br><font size=2 face="Arial"> fileid:mapping
= fsname</font>
<br><font size=2 face="Arial"> gpfs:sharemodes
= yes</font>
<br><font size=2 face="Arial"> gpfs:leases
= yes</font>
<br><font size=2 face="Arial"> gpfs:dfreequota
= yes</font>
<br><font size=2 face="Arial"> gpfs:hsm
= yes</font>
<br><font size=2 face="Arial"> syncops:onmeta
= no</font>
<br><font size=2 face="Arial"> kernel
oplocks = no</font>
<br><font size=2 face="Arial"> level2
oplocks = yes</font>
<br><font size=2 face="Arial"> notify:inotify
= no</font>
<br><font size=2 face="Arial"> vfs objects
= shadow_copy2 syncops gpfs fileid</font>
<br><font size=2 face="Arial"> shadow:snapdir
= .snapshots</font>
<br><font size=2 face="Arial"> shadow:fixinodes
= yes</font>
<br><font size=2 face="Arial"> shadow:snapdirseverywhere
= yes</font>
<br><font size=2 face="Arial"> shadow:sort
= desc</font>
<br><font size=2 face="Arial"> wide
links = no</font>
<br><font size=2 face="Arial"> async
smb echo handler = yes</font>
<br><font size=2 face="Arial"> smbd:backgroundqueue
= False</font>
<br><font size=2 face="Arial"> use sendfile
= no</font>
<br><font size=2 face="Arial"> strict
locking = yes</font>
<br><font size=2 face="Arial"> posix
locking = yes</font>
<br><font size=2 face="Arial"> force
unknown acl user = yes</font>
<br><font size=2 face="Arial"> nfs4:mode
= simple</font>
<br><font size=2 face="Arial"> nfs4:chown
= yes</font>
<br><font size=2 face="Arial"> nfs4:acedup
= merge</font>
<br><font size=2 face="Arial"> nfs4:sidmap
= /etc/samba/sidmap.tdb</font>
<br><font size=2 face="Arial"> gpfs:winattr
= yes</font>
<br><font size=2 face="Arial"> store
dos attributes = yes</font>
<br><font size=2 face="Arial"> map readonly
= yes</font>
<br><font size=2 face="Arial"> map archive
= yes</font>
<br><font size=2 face="Arial"> map system
= yes</font>
<br><font size=2 face="Arial"> map hidden
= yes</font>
<br><font size=2 face="Arial"> ea support
= yes</font>
<br><font size=2 face="Arial"> dmapi
support = no</font>
<br><font size=2 face="Arial"> unix
extensions = no</font>
<br><font size=2 face="Arial"> socket
options = TCP_NODELAY SO_KEEPALIVE TCP_KEEPCNT=4 TCP_KEEPIDLE=240 TCP_KEEPINTVL=15</font>
<br><font size=2 face="Arial"> strict
allocate = yes</font>
<br><font size=2 face="Arial"> gpfs:prealloc
= yes</font>
<br>
<br>
<br><font size=2 face="Arial">if you don't configure using the registry
you need to maintain the smb.conf file on all your nodes and i am not diving
into how to setup the registry, but for the people who care, michael adam's
presentation is a good starting point </font><a href="http://www.samba.org/~obnox/presentations/linux-kongress-2008/lk2008-obnox.pdf"><font size=2 color=blue face="Arial"><u>http://www.samba.org/~obnox/presentations/linux-kongress-2008/lk2008-obnox.pdf</u></font></a><font size=2 face="Arial">
</font>
<br>
<br><font size=2 face="Arial">also depending on your authentication/idmap
setup , there are multiple changes/additions that needs to be made.</font>
<br>
<br><font size=2 face="Arial">on the gpfs side you should set the following
config parameters : </font>
<br>
<br><font size=2 face="Arial">cifsBypassShareLocksOnRename=yes</font>
<br><font size=2 face="Arial">syncSambaMetadataOps=yes</font>
<br><font size=2 face="Arial">cifsBypassTraversalChecking=yes</font>
<br><font size=2 face="Arial">allowSambaCaseInsensitiveLookup=no</font>
<br>
<br><font size=2 face="Arial">your filesystem should have the following
settings configured :</font>
<br>
<br><font size=2 face="Arial"> -D
nfs4
File locking semantics in effect</font>
<br><font size=2 face="Arial"> -k
nfs4
ACL semantics in effect</font>
<br><font size=2 face="Arial"> -o
nfssync
Additional mount options</font>
<br><font size=2 face="Arial"> --fastea
Yes
Fast external attributes enabled?</font>
<br><font size=2 face="Arial"> -S
relatime
Suppress atime mount option</font>
<br>
<br><font size=2 face="Arial">the -S is a performance optimization, but
you need to check if your backup/av or other software can deal with it,
it essentially reduces the frequency of atime updates to once every 24
hours which is the new default on nfs mounts and others as well. </font>
<br>
<br><font size=2 face="Arial">a lot of the configuration parameters above
are also above and beyond locking and data integrity, they are for better
windows compatibility and should be checked if applicable for your environment.</font>
<br>
<br><font size=2 face="Arial">i would also recommend to run on GPFS 3.5
TL3 or newer to get the proper GPFS level of fixes for this type of configurations.
</font>
<br>
<br><font size=2 face="Arial">i would like to repeat that i don't write
this email to encourage people to all go start installing/ configuring
samba on top of GPFS as i pointed out that you are kind on your own unless
you can read source code and/or have somebody who does and is able to help
as soon as you run into a problem. </font>
<br><font size=2 face="Arial">the main point of sharing this information
is to clarify a lot of misinformation out there and provide the people
who already have setups that are incorrect the information to at least
not run into data corruption issues do to wrong configuration. </font>
<br>
<br><font size=2 face="Arial">Sven</font>
<br>
<br>
<br>
<br>
<br><font size=2 color=#5f5f5f face="Arial">From:
</font><font size=2 face="Arial">Chair <chair@gpfsug.org></font>
<br><font size=2 color=#5f5f5f face="Arial">To: </font><font size=2 face="Arial">gpfsug-discuss@gpfsug.org</font>
<br><font size=2 color=#5f5f5f face="Arial">Date:
</font><font size=2 face="Arial">12/16/2013 03:31 AM</font>
<br><font size=2 color=#5f5f5f face="Arial">Subject:
</font><font size=2 face="Arial">Re: [gpfsug-discuss] GPFS
and both Samba and NFS</font>
<br><font size=2 color=#5f5f5f face="Arial">Sent by:
</font><font size=2 face="Arial">gpfsug-discuss-bounces@gpfsug.org</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=2 face="Arial">Allo<br>
<br>
Just jumping in here a minute:<br>
<br>
> It is unworthy of an IBM employee to spread such inaccurate misinformation.<br>
<br>
Whilst this may be inaccurate - I very, very, much doubt that IBM or <br>
their employees have a secret master plan to spread misinformation (!)<br>
In the spirit of this group, let's work together to technically look at
<br>
such issues.<br>
<br>
Sven, if that is the case, perhaps you could crib the lines of code / <br>
show your methodology that supports your views / experience.<br>
<br>
<br>
Regards,<br>
<br>
Jez<br>
--<br>
UG Chair<br>
<br>
<br>
<br>
On 16/12/13 11:21, Jonathan Buzzard wrote:<br>
> On Fri, 2013-12-13 at 10:14 -0800, Sven Oehme wrote:<br>
><br>
> [SNIP]<br>
><br>
>> the only way to get something working (don't get confused with<br>
>> officially Supported) is to recompile the CTDB src packages AND
the<br>
>> Samba src packages on a node that has GPFS already installed.
also the<br>
>> inclusion of CTDB into Samba will not address this, its just a
more<br>
>> convenient packaging.<br>
>><br>
>> Only if the build happens on such a node things like the vfs modules<br>
>> for GPFS are build and included in the package.<br>
>><br>
> That is a factually inaccurate statement. There is nothing in CTDB
that<br>
> is GPFS specific. Trust me I have examined the code closely to determine<br>
> if this is the case. So unless this has changed recently you are flat<br>
> out wrong.<br>
><br>
> Consequently there is no requirement whatsoever to rebuild CTDB to
get<br>
> the vfs_gpfs module. In addition there is also no requirement to<br>
> actually have GPFS installed to build the vfs_gpfs module either.
What<br>
> you need to have is the GPFS GPL header files and nothing else. As
it is<br>
> a loadable VFS module linking takes place at load time not compile
time.<br>
><br>
> It is unworthy of an IBM employee to spread such inaccurate<br>
> misinformation.<br>
><br>
> [SNIP]<br>
><br>
>> said all this the binaries alone are only part of the Solution,
after<br>
>> you have the correct packages, you need to properly configuration
the<br>
>> system and setting all the right options (on GPFS as well as on
CTDB<br>
>> and smbd.conf), which unfortunate are very System configuration<br>
>> specific, as otherwise you still can end up with data corruption
if<br>
>> not set right.<br>
> Indeed. However I know not only what those options are, but also what<br>
> they do despite IBM's refusal to tell us anything about them.<br>
><br>
> I would also point out that there are sites that where running Samba
on<br>
> top of GPFS for many years before IBM began offering their<br>
> SONAS/Storwize Unifed products.<br>
><br>
><br>
> JAB.<br>
><br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<br>
</font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><font size=2 face="Arial">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></a><font size=2 face="Arial"><br>
<br>
</font>
<br>