[gpfsug-discuss] unified file and object

Bill Owen billowen at us.ibm.com
Fri Oct 28 15:37:27 BST 2016


Hi Leslie,
For the issues you list below:
1.  AD configuration - there is a known issue with quoted names passed to
mmuserauth - we are investigating how to correct this.
2.  Can you provide more details on how you configured file access?  The
normal procedure is to use "mmobj file-access enable", and this will set up
the required settings in the config file.  Can you send us:
- the steps used to configure file access
- the resulting /etc/swift/object-server-sof.conf
- log files from /var/log/swift or output of  "systemctl status
openstack-swift-object-sof"

We can schedule a short call to help debug if needed.

Here is a more detailed description of enabling file access to object data:

Enable unified file and object access
See:
http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.1/com.ibm.spectrum.scale.v4r21.doc/bl1adm_enablefileaccess.htm


1. enable & configure:
# mmobj file-access enable

# mmobj config list --ccrfile spectrum-scale-object.conf --section
capabilities --property file-access-enabled
file-access-enabled = true

Different user id mapping approaches are supporting.  The simplest is
id_mgmt = local_mode.  This example leaves that default in place.

Verify id_mgmt mode:
# mmobj config list --ccrfile object-server-sof.conf --section DEFAULT
--property id_mgmt
id_mgmt = local_mode

At this point, you can create a storage policy that will be used for file &
object access containers.  We'll create a new fileset for this step (in
4.2.2 we will support objectizing existing filesets).  In this example, we
create the fileset with 1M inodes initially allocated:
# mmobj policy create fileaccess --enable-file-access  -i 1000000
[I] Getting latest configuration from ccr
[I] Creating fileset fs2-1m-03:obj_fileaccess
[I] Creating new unique index and building the object rings
[I] Updating the configuration
[I] Uploading the changed configuration

The new storage policy is listed:
# mmobj policy list

Index       Name       Default Deprecated Fileset        Functions
--------------------------------------------------------------------------------
0           policy-0   yes                Object_Fileset	default
87811608170 fileaccess                    obj_fileaccess
file-and-object-access


The new fileset called obj_fileaccess is shown below:
# mmlsfileset fs2-1m-03 obj_fileaccess -L
Filesets in file system 'fs2-1m-03':
Name                            Id      RootInode  ParentId Created
InodeSpace      MaxInodes    AllocInodes Comment
obj_fileaccess                   6      134217731         0 Wed Aug 17
10:08:51 2016        4              1000192        1000192

2.  Create one or more containers that are file-access enabled:
Next, you can create one or more containers that use this storage policy
(and have file access enabled):
# source openrc
# swift post fileaccess_container --header "x-storage-policy: fileaccess"
# swift stat fileaccess_container
         Account: AUTH_acad33df8cdf402ebab6bfb87b1674af
       Container: fileaccess_container
         Objects: 0
           Bytes: 0
        Read ACL:
       Write ACL:
         Sync To:
        Sync Key:
   Accept-Ranges: bytes
X-Storage-Policy: fileaccess
     X-Timestamp: 1471456287.21609
      X-Trans-Id: tx0f305353a20a4ac1ab927-0057b4a428
    Content-Type: text/plain; charset=utf-8

Objects can be added to the container from the object or file interface,
the file path is structured like this:  <mount
point>/<fileset>/<device>/<account>/<container>/

In our example it would be:
/ibm/fs2-1m-03/obj_fileaccess/s87811608170z1device1/AUTH_acad33df8cdf402ebab6bfb87b1674af/fileaccess_container/

For convenience, create a soft link:
ln
-s /ibm/fs2-1m-03/obj_fileaccess/s87811608170z1device1/AUTH_acad33df8cdf402ebab6bfb87b1674af/fileaccess_container/ /ibm/fs2-1m-03/unified_file_object

3.  Verify access from both file & object interface:
Upload a file from object interface, it will be readable from file
interface, owned by swift user.  For example:
# date > test1.obj
# swift upload fileaccess_container  test1.obj
test1.obj

# ls -l /ibm/fs2-1m-03/unified_file_object/
total 0
-rwxr-xr-x 1 swift swift 29 Aug 17 11:03 test1.obj


Similarly, create an example object (or set of objects) from file
interface:
# date > /ibm/fs2-1m-03/unified_file_object/test2.obj

The object must be readable by swift user - either by setting file
ownership or using file acls:
# chown swift:swift /ibm/fs2-1m-03/unified_file_object/test2.obj
# ls -l /ibm/fs2-1m-03/unified_file_object/
-rwxr-xr-x 1 swift swift 29 Aug 17 11:03 test1.obj
-rw-r--r-- 1 swift swift 29 Aug 17 11:05 test2.obj

New files will be "objectized" (made visible to the object interface) by a
periodic process.  The default period for this is 30 min, but can be
changed using mmob config change command.  This process can be resource
intensive, so don't see to run too often if there are many containers &
files.

# mmobj config list --ccrfile spectrum-scale-objectizer.conf --section
DEFAULT --property objectization_interval
objectization_interval = 1800

After waiting the objectizer interval, the new object shows up from the
object interface, and can be accessed like any other object:
# swift list fileaccess_container
test1.obj
test2.obj

# swift download fileaccess_container test2.obj
test2.obj [auth 0.229s, headers 0.862s, total 0.862s, 0.000 MB/s]

Verify the contents match
# cat test2.obj
Wed Aug 17 11:05:00 PDT 2016

# cat /ibm/fs2-1m-03/unified_file_object/test2.obj
Wed Aug 17 11:05:00 PDT 2016

And update from file interface:
# echo "bill was here" >>  /ibm/fs2-1m-03/unified_file_object/test2.obj

Redownload from object interface and confirm:
# swift download fileaccess_container test2.obj
test2.obj [auth 0.230s, headers 0.729s, total 0.729s, 0.000 MB/s]
[
# cat test2.obj
Wed Aug 17 11:05:00 PDT 2016
bill was here

Use Case Example:
One usecase is adding/expanding an archive file in file interface, and
results can be accessed from file interface.

For example, create a set of test files:
# mkdir test
[root at client22 ~]# for i in {1..10}; do  date > test/smallobj$i ; done
[root at client22 ~]# ll test
total 40
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj1
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj10
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj2
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj3
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj4
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj5
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj6
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj7
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj8
-rw-r--r-- 1 root root 29 Aug 17 11:31 smallobj9
# tar -cf databag.tar test

Expand the file into file interface and set ownership to swift:
# cd /ibm/fs2-1m-03/unified_file_object/
# tar -xf ~/databag.tar
# chown -R swift:swift test

After objectizer runs, all files are available from object interface:
# swift list fileaccess_container
test/smallobj1
test/smallobj10
test/smallobj2
test/smallobj3
test/smallobj4
test/smallobj5
test/smallobj6
test/smallobj7
test/smallobj8
test/smallobj9
test1.obj
test2.obj

Regards,
Bill Owen
billowen at us.ibm.com
Spectrum Scale Object Storage
520-799-4829




From:	leslie elliott <leslie.james.elliott at gmail.com>
To:	gpfsug-discuss at spectrumscale.org
Date:	10/25/2016 03:01 PM
Subject:	[gpfsug-discuss] unified file and object
Sent by:	gpfsug-discuss-bounces at spectrumscale.org



Hi

We are in the process of trying to configure unified file and object
storage in unified_mode and have a few problems

We are running 4.2.1 and do not have any current issues the file access
protocols setting up the authentication

First issue we do have is binding the object data to our Active Directory,
we seem to be hitting a road block due to the fact that the bind DN has
spaces in it, if we enclose the DN in quotes it still fails, if we escape
them with the appropriate RFC  value we can get the mmuserauth to complete
but the lookups from the local keystone fail for the authentication of the
users

The DN for the swift user and swift admin also have quotes in them, so just
doing it on the command line is not enough to get the mmuserauth command to
complete

Second problem is

OBJ:openstack-swift-object-sof           is not running

This seems to be due to the config file not having  bind_ip and bind_port
values, if these are added then the error turns to pipeline of other
settings in the config file missing

This particular issue occurs no matter what the auth type is set to be for
object

Hopefully this make some sense to someone

Thanks

leslie


Leslie Elliott, Infrastructure Support Specialist,  Faculty Infrastructure
and Applications Support
Information Technology Services, The University of Queensland
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20161028/e6e02153/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20161028/e6e02153/attachment-0002.gif>


More information about the gpfsug-discuss mailing list