[gpfsug-discuss] Copying ACLs between filesystems

orlando.richards at ed.ac.uk orlando.richards at ed.ac.uk
Mon Oct 1 11:07:56 BST 2012 

On Mon, 1 Oct 2012, Jez Tucker wrote:

> I'd put the tarball, patch and binary.
> That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src.
>
> Let me know your github user and I'll give you r/w.

OrlandoRichards (orlando.richards at ed.ac.uk)

>
>> -----Original Message-----
>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss-
>> bounces at gpfsug.org] On Behalf Of Orlando Richards
>> Sent: 01 October 2012 10:42
>> To: gpfsug main discussion list
>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems
>>
>> On 01/10/12 10:33, Jez Tucker wrote:
>>> Nice.  Can you put it on the github?
>>>
>>> https://github.com/gpfsug/gpfsug-tools
>>>
>>
>> Sure - the whole rsync source, or just the patch?
>>
>> Also - I've discovered that it won't identify differences between ACLs to
>> trigger a copy. When it does copy, it copies the ACLs though.
>>
>> For background - we're thinking of using this for building an asynchronously-
>> synced DR filesystem, which is an HSM filesystem (as opposed to the
>> production filesystem, which is disk only). Thus we'll be using GPFS to find
>> candidate files (based on CTIME and/or MTIME) and passing the list to rsync
>> to sync over. So we could pass the list to "touch" to touch the destination
>> files before syncing to them, to force rsync to do a copy.
>>
>> --
>> Orlando
>>
>>>
>>>> -----Original Message-----
>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss-
>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards
>>>> Sent: 01 October 2012 10:20
>>>> To: gpfsug-discuss at gpfsug.org
>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems
>>>>
>>>> Whoop! Got it!
>>>>
>>>> Attached is the complete patch against the rsync-3.0.9 release bundle:
>>>>    http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
>>>>
>>>> The patch was taken with:
>>>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/  > gpfs_support_rsync-
>>>> 3.0.9.patch
>>>>
>>>> To build once patched - do:
>>>> ./configure --enable-acl-support --enable-xattr-support
>>>> --enable-gpfs- support
>>>>
>>>> (make sure that libacl1-dev, or equivalent, is available).
>>>>
>>>> Seems to work as desired.
>>>>
>>>> In the end, I removed all the "if DEBUG_GTE" statements from the
>>>> gpfs.c code to get it working (!).
>>>>
>>>> --
>>>> Orlando
>>>>
>>>> On 28/09/12 15:46, Jez Tucker wrote:
>>>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr
>>>>> issues all
>>>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a
>> punt.
>>>>> Grabbing rsync latest git.
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss-
>>>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards
>>>>>> Sent: 28 September 2012 15:11
>>>>>> To: gpfsug-discuss at gpfsug.org
>>>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems
>>>>>>
>>>>>> On 28/09/12 12:30, Jonathan Buzzard wrote:
>>>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote:
>>>>>>>> On 28/09/12 11:28, Barry Evans wrote:
>>>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either.
>>>>>>>>> two filesystems, same machine.
>>>>>>>>>
>>>>>>>>> sounds very, very evil but have you tried robocopy from one
>>>>>>>>> share to another? Not ideal.
>>>>>>>>
>>>>>>>> I've considered it - but not yet desperate enough to try it :)
>>>>>>>>
>>>>>>>
>>>>>>> When I have tried doing this from Windows file servers the results
>>>>>>> where not nice as Windows has ACL's for things like administrators
>>>>>>> etc. that robocopy attempted to copy over. It was also painfully slow.
>>>>>>> I found xcopy was a lot faster, though as my source was a Windows
>>>>>>> server the resultant ACL's were still borked.
>>>>>>>
>>>>>>>> Still looking for something sane.
>>>>>>>>
>>>>>>>
>>>>>>> Would AFM not do the trick?
>>>>>>>
>>>>>>>> Good to see I'm not just missing something blatantly obvious
>> though!
>>>>>>>>
>>>>>>>
>>>>>>> What's your coding like? Depending how often you are going to be
>>>>>>> attempting this I would fix the IBM patches for rsync so they work
>>>>>>> on a modern version of rsync. Patch does not look to complicated to
>> me.
>>>>>>>
>>>>>>
>>>>>> Beyond my abilities it seems. If anyone fancies having a friday
>>>>>> afternoon coding challenge, here's the problem:
>>>>>>
>>>>>> I'm trying to apply this patch:
>>>>>>
>>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/
>>>>>> rs
>>>>>> ync
>>>>>> _gpfs_attrs-git.obj
>>>>>>
>>>>>> to rsync - let's say version 3.0.6:
>>>>>>
>>>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz
>>>>>>
>>>>>> So I do:
>>>>>> mkdir -p src/rsync
>>>>>> cd src/rsync
>>>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz
>>>>>> wget
>>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/
>>>>>> rs
>>>>>> ync
>>>>>> _gpfs_attrs-git.obj
>>>>>> tar -zxf rsync-3.0.6.tar.gz
>>>>>> cd rsync-3.0.6
>>>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff
>>>>>>
>>>>>> ./configure --prefix=/tmp
>>>>>> make
>>>>>>
>>>>>>
>>>>>> Error message is:
>>>>>>
>>>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W  -c
>>>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0,
>>>>>>                     from gpfs.c:20:
>>>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside
>>>>>> parameter list [enabled by default]
>>>>>> proto.h:111:28: warning: its scope is only this definition or
>>>>>> declaration, which is probably not what you want [enabled by
>>>>>> default]
>>>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr'
>>>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was
>>>>>> here
>>>>>> gpfs.c: In function 'gpfs_free_sxp':
>>>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c: In function 'gpfs_free_list':
>>>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE'
>>>>>> [-Wimplicit-function-declaration]
>>>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c:162:17: note: each undeclared identifier is reported only
>>>>>> once for each function it appears in
>>>>>> gpfs.c: In function 'gpfs_receive_attr_int':
>>>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c: In function 'gpfs_find_attr':
>>>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c: In function 'gpfs_cache_attr':
>>>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c: In function 'gpfs_send_attr':
>>>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c: In function 'gpfs_get_attr':
>>>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c: In function 'gpfs_set_attr':
>>>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function)
>>>>>> gpfs.c: In function 'gpfs_attr_get_changed':
>>>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr'
>>>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function)
>>>>>> make: *** [gpfs.o] Error 1
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>                --
>>>>>>       Dr Orlando Richards
>>>>>>      Information Services
>>>>>> IT Infrastructure Division
>>>>>>           Unix Section
>>>>>>        Tel: 0131 650 4994
>>>>>>
>>>>>> The University of Edinburgh is a charitable body, registered in
>>>>>> Scotland, with registration number SC005336.
>>>>>> _______________________________________________
>>>>>> gpfsug-discuss mailing list
>>>>>> gpfsug-discuss at gpfsug.org
>>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> gpfsug-discuss mailing list
>>>>> gpfsug-discuss at gpfsug.org
>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>>>
>>>>
>>>>
>>>> --
>>>>               --
>>>>      Dr Orlando Richards
>>>>     Information Services
>>>> IT Infrastructure Division
>>>>          Unix Section
>>>>       Tel: 0131 650 4994
>>>>
>>>> The University of Edinburgh is a charitable body, registered in
>>>> Scotland, with registration number SC005336.
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss at gpfsug.org
>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>
>>
>>
>> --
>>              --
>>     Dr Orlando Richards
>>    Information Services
>> IT Infrastructure Division
>>         Unix Section
>>      Tel: 0131 650 4994
>>
>> The University of Edinburgh is a charitable body, registered in Scotland, with
>> registration number SC005336.
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at gpfsug.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>

-- 
             --
    Dr Orlando Richards
   Information Services
IT Infrastructure Division
        Unix Section
     Tel: 0131 650 4994

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

More information about the gpfsug-discuss mailing list