From orlando.richards at ed.ac.uk Mon Oct 1 10:20:23 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:20:23 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696057.3080005@ed.ac.uk> Whoop! Got it! Attached is the complete patch against the rsync-3.0.9 release bundle: http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz The patch was taken with: diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync-3.0.9.patch To build once patched - do: ./configure --enable-acl-support --enable-xattr-support --enable-gpfs-support (make sure that libacl1-dev, or equivalent, is available). Seems to work as desired. In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c code to get it working (!). -- Orlando On 28/09/12 15:46, Jez Tucker wrote: > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > Grabbing rsync latest git. > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 28 September 2012 15:11 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 28/09/12 12:30, Jonathan Buzzard wrote: >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>> On 28/09/12 11:28, Barry Evans wrote: >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>> two filesystems, same machine. >>>>> >>>>> sounds very, very evil but have you tried robocopy from one share to >>>>> another? Not ideal. >>>> >>>> I've considered it - but not yet desperate enough to try it :) >>>> >>> >>> When I have tried doing this from Windows file servers the results >>> where not nice as Windows has ACL's for things like administrators >>> etc. that robocopy attempted to copy over. It was also painfully slow. >>> I found xcopy was a lot faster, though as my source was a Windows >>> server the resultant ACL's were still borked. >>> >>>> Still looking for something sane. >>>> >>> >>> Would AFM not do the trick? >>> >>>> Good to see I'm not just missing something blatantly obvious though! >>>> >>> >>> What's your coding like? Depending how often you are going to be >>> attempting this I would fix the IBM patches for rsync so they work on >>> a modern version of rsync. Patch does not look to complicated to me. >>> >> >> Beyond my abilities it seems. If anyone fancies having a friday afternoon >> coding challenge, here's the problem: >> >> I'm trying to apply this patch: >> >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> >> to rsync - let's say version 3.0.6: >> >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> >> So I do: >> mkdir -p src/rsync >> cd src/rsync >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> wget >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> tar -zxf rsync-3.0.6.tar.gz >> cd rsync-3.0.6 >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >> >> ./configure --prefix=/tmp >> make >> >> >> Error message is: >> >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c gpfs.c -o >> gpfs.o In file included from rsync.h:920:0, >> from gpfs.c:20: >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside parameter >> list [enabled by default] >> proto.h:111:28: warning: its scope is only this definition or declaration, >> which is probably not what you want [enabled by default] >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was here >> gpfs.c: In function 'gpfs_free_sxp': >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_free_list': >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >> [-Wimplicit-function-declaration] >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c:162:17: note: each undeclared identifier is reported only once for >> each function it appears in >> gpfs.c: In function 'gpfs_receive_attr_int': >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_find_attr': >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_cache_attr': >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_send_attr': >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_get_attr': >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_set_attr': >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_attr_get_changed': >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >> make: *** [gpfs.o] Error 1 >> >> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -------------- next part -------------- A non-text attachment was scrubbed... Name: gpfs_support_rsync-3.0.9.patch Type: text/x-patch Size: 33668 bytes Desc: not available URL: From Jez.Tucker at rushes.co.uk Mon Oct 1 10:33:42 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:33:42 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696057.3080005@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Nice. Can you put it on the github? https://github.com/gpfsug/gpfsug-tools > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:20 > To: gpfsug-discuss at gpfsug.org > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > Whoop! Got it! > > Attached is the complete patch against the rsync-3.0.9 release bundle: > http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > > The patch was taken with: > diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > 3.0.9.patch > > To build once patched - do: > ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- > support > > (make sure that libacl1-dev, or equivalent, is available). > > Seems to work as desired. > > In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c > code to get it working (!). > > -- > Orlando > > On 28/09/12 15:46, Jez Tucker wrote: > > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all > the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > > Grabbing rsync latest git. > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 28 September 2012 15:11 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>> two filesystems, same machine. > >>>>> > >>>>> sounds very, very evil but have you tried robocopy from one share > >>>>> to another? Not ideal. > >>>> > >>>> I've considered it - but not yet desperate enough to try it :) > >>>> > >>> > >>> When I have tried doing this from Windows file servers the results > >>> where not nice as Windows has ACL's for things like administrators > >>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>> I found xcopy was a lot faster, though as my source was a Windows > >>> server the resultant ACL's were still borked. > >>> > >>>> Still looking for something sane. > >>>> > >>> > >>> Would AFM not do the trick? > >>> > >>>> Good to see I'm not just missing something blatantly obvious though! > >>>> > >>> > >>> What's your coding like? Depending how often you are going to be > >>> attempting this I would fix the IBM patches for rsync so they work > >>> on a modern version of rsync. Patch does not look to complicated to me. > >>> > >> > >> Beyond my abilities it seems. If anyone fancies having a friday > >> afternoon coding challenge, here's the problem: > >> > >> I'm trying to apply this patch: > >> > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> > >> to rsync - let's say version 3.0.6: > >> > >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> > >> So I do: > >> mkdir -p src/rsync > >> cd src/rsync > >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> wget > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> tar -zxf rsync-3.0.6.tar.gz > >> cd rsync-3.0.6 > >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >> > >> ./configure --prefix=/tmp > >> make > >> > >> > >> Error message is: > >> > >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >> from gpfs.c:20: > >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >> parameter list [enabled by default] > >> proto.h:111:28: warning: its scope is only this definition or > >> declaration, which is probably not what you want [enabled by default] > >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >> here > >> gpfs.c: In function 'gpfs_free_sxp': > >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_free_list': > >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >> [-Wimplicit-function-declaration] > >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c:162:17: note: each undeclared identifier is reported only once > >> for each function it appears in > >> gpfs.c: In function 'gpfs_receive_attr_int': > >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_find_attr': > >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_cache_attr': > >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_send_attr': > >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_get_attr': > >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_set_attr': > >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_attr_get_changed': > >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >> make: *** [gpfs.o] Error 1 > >> > >> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > >> _______________________________________________ > >> gpfsug-discuss mailing list > >> gpfsug-discuss at gpfsug.org > >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. From orlando.richards at ed.ac.uk Mon Oct 1 10:41:55 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:41:55 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696563.3000507@ed.ac.uk> On 01/10/12 10:33, Jez Tucker wrote: > Nice. Can you put it on the github? > > https://github.com/gpfsug/gpfsug-tools > Sure - the whole rsync source, or just the patch? Also - I've discovered that it won't identify differences between ACLs to trigger a copy. When it does copy, it copies the ACLs though. For background - we're thinking of using this for building an asynchronously-synced DR filesystem, which is an HSM filesystem (as opposed to the production filesystem, which is disk only). Thus we'll be using GPFS to find candidate files (based on CTIME and/or MTIME) and passing the list to rsync to sync over. So we could pass the list to "touch" to touch the destination files before syncing to them, to force rsync to do a copy. -- Orlando > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:20 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> Whoop! Got it! >> >> Attached is the complete patch against the rsync-3.0.9 release bundle: >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >> >> The patch was taken with: >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >> 3.0.9.patch >> >> To build once patched - do: >> ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- >> support >> >> (make sure that libacl1-dev, or equivalent, is available). >> >> Seems to work as desired. >> >> In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c >> code to get it working (!). >> >> -- >> Orlando >> >> On 28/09/12 15:46, Jez Tucker wrote: >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. >>> Grabbing rsync latest git. >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 28 September 2012 15:11 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>> two filesystems, same machine. >>>>>>> >>>>>>> sounds very, very evil but have you tried robocopy from one share >>>>>>> to another? Not ideal. >>>>>> >>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>> >>>>> >>>>> When I have tried doing this from Windows file servers the results >>>>> where not nice as Windows has ACL's for things like administrators >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>> server the resultant ACL's were still borked. >>>>> >>>>>> Still looking for something sane. >>>>>> >>>>> >>>>> Would AFM not do the trick? >>>>> >>>>>> Good to see I'm not just missing something blatantly obvious though! >>>>>> >>>>> >>>>> What's your coding like? Depending how often you are going to be >>>>> attempting this I would fix the IBM patches for rsync so they work >>>>> on a modern version of rsync. Patch does not look to complicated to me. >>>>> >>>> >>>> Beyond my abilities it seems. If anyone fancies having a friday >>>> afternoon coding challenge, here's the problem: >>>> >>>> I'm trying to apply this patch: >>>> >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> >>>> to rsync - let's say version 3.0.6: >>>> >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> >>>> So I do: >>>> mkdir -p src/rsync >>>> cd src/rsync >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> wget >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> tar -zxf rsync-3.0.6.tar.gz >>>> cd rsync-3.0.6 >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>> >>>> ./configure --prefix=/tmp >>>> make >>>> >>>> >>>> Error message is: >>>> >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>> from gpfs.c:20: >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>> parameter list [enabled by default] >>>> proto.h:111:28: warning: its scope is only this definition or >>>> declaration, which is probably not what you want [enabled by default] >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>> here >>>> gpfs.c: In function 'gpfs_free_sxp': >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_free_list': >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>> [-Wimplicit-function-declaration] >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c:162:17: note: each undeclared identifier is reported only once >>>> for each function it appears in >>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_find_attr': >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_cache_attr': >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_send_attr': >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_get_attr': >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_set_attr': >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>> make: *** [gpfs.o] Error 1 >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> gpfsug-discuss mailing list >>>> gpfsug-discuss at gpfsug.org >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 1 10:47:13 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:47:13 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696563.3000507@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> I'd put the tarball, patch and binary. That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. Let me know your github user and I'll give you r/w. > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:42 > To: gpfsug main discussion list > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > On 01/10/12 10:33, Jez Tucker wrote: > > Nice. Can you put it on the github? > > > > https://github.com/gpfsug/gpfsug-tools > > > > Sure - the whole rsync source, or just the patch? > > Also - I've discovered that it won't identify differences between ACLs to > trigger a copy. When it does copy, it copies the ACLs though. > > For background - we're thinking of using this for building an asynchronously- > synced DR filesystem, which is an HSM filesystem (as opposed to the > production filesystem, which is disk only). Thus we'll be using GPFS to find > candidate files (based on CTIME and/or MTIME) and passing the list to rsync > to sync over. So we could pass the list to "touch" to touch the destination > files before syncing to them, to force rsync to do a copy. > > -- > Orlando > > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 01 October 2012 10:20 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> Whoop! Got it! > >> > >> Attached is the complete patch against the rsync-3.0.9 release bundle: > >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > >> > >> The patch was taken with: > >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > >> 3.0.9.patch > >> > >> To build once patched - do: > >> ./configure --enable-acl-support --enable-xattr-support > >> --enable-gpfs- support > >> > >> (make sure that libacl1-dev, or equivalent, is available). > >> > >> Seems to work as desired. > >> > >> In the end, I removed all the "if DEBUG_GTE" statements from the > >> gpfs.c code to get it working (!). > >> > >> -- > >> Orlando > >> > >> On 28/09/12 15:46, Jez Tucker wrote: > >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr > >>> issues all > >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a > punt. > >>> Grabbing rsync latest git. > >>> > >>>> -----Original Message----- > >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards > >>>> Sent: 28 September 2012 15:11 > >>>> To: gpfsug-discuss at gpfsug.org > >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >>>> > >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>>>> two filesystems, same machine. > >>>>>>> > >>>>>>> sounds very, very evil but have you tried robocopy from one > >>>>>>> share to another? Not ideal. > >>>>>> > >>>>>> I've considered it - but not yet desperate enough to try it :) > >>>>>> > >>>>> > >>>>> When I have tried doing this from Windows file servers the results > >>>>> where not nice as Windows has ACL's for things like administrators > >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>>>> I found xcopy was a lot faster, though as my source was a Windows > >>>>> server the resultant ACL's were still borked. > >>>>> > >>>>>> Still looking for something sane. > >>>>>> > >>>>> > >>>>> Would AFM not do the trick? > >>>>> > >>>>>> Good to see I'm not just missing something blatantly obvious > though! > >>>>>> > >>>>> > >>>>> What's your coding like? Depending how often you are going to be > >>>>> attempting this I would fix the IBM patches for rsync so they work > >>>>> on a modern version of rsync. Patch does not look to complicated to > me. > >>>>> > >>>> > >>>> Beyond my abilities it seems. If anyone fancies having a friday > >>>> afternoon coding challenge, here's the problem: > >>>> > >>>> I'm trying to apply this patch: > >>>> > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> > >>>> to rsync - let's say version 3.0.6: > >>>> > >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> > >>>> So I do: > >>>> mkdir -p src/rsync > >>>> cd src/rsync > >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> wget > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> tar -zxf rsync-3.0.6.tar.gz > >>>> cd rsync-3.0.6 > >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >>>> > >>>> ./configure --prefix=/tmp > >>>> make > >>>> > >>>> > >>>> Error message is: > >>>> > >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >>>> from gpfs.c:20: > >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >>>> parameter list [enabled by default] > >>>> proto.h:111:28: warning: its scope is only this definition or > >>>> declaration, which is probably not what you want [enabled by > >>>> default] > >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >>>> here > >>>> gpfs.c: In function 'gpfs_free_sxp': > >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_free_list': > >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >>>> [-Wimplicit-function-declaration] > >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c:162:17: note: each undeclared identifier is reported only > >>>> once for each function it appears in > >>>> gpfs.c: In function 'gpfs_receive_attr_int': > >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_find_attr': > >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_cache_attr': > >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_send_attr': > >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_get_attr': > >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_set_attr': > >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_attr_get_changed': > >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >>>> make: *** [gpfs.o] Error 1 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> -- > >>>> Dr Orlando Richards > >>>> Information Services > >>>> IT Infrastructure Division > >>>> Unix Section > >>>> Tel: 0131 650 4994 > >>>> > >>>> The University of Edinburgh is a charitable body, registered in > >>>> Scotland, with registration number SC005336. > >>>> _______________________________________________ > >>>> gpfsug-discuss mailing list > >>>> gpfsug-discuss at gpfsug.org > >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >>> > >>> _______________________________________________ > >>> gpfsug-discuss mailing list > >>> gpfsug-discuss at gpfsug.org > >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss From orlando.richards at ed.ac.uk Mon Oct 1 11:07:56 2012 From: orlando.richards at ed.ac.uk (orlando.richards at ed.ac.uk) Date: Mon, 1 Oct 2012 11:07:56 +0100 (BST) Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> Message-ID: On Mon, 1 Oct 2012, Jez Tucker wrote: > I'd put the tarball, patch and binary. > That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. > > Let me know your github user and I'll give you r/w. OrlandoRichards (orlando.richards at ed.ac.uk) > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:42 >> To: gpfsug main discussion list >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 01/10/12 10:33, Jez Tucker wrote: >>> Nice. Can you put it on the github? >>> >>> https://github.com/gpfsug/gpfsug-tools >>> >> >> Sure - the whole rsync source, or just the patch? >> >> Also - I've discovered that it won't identify differences between ACLs to >> trigger a copy. When it does copy, it copies the ACLs though. >> >> For background - we're thinking of using this for building an asynchronously- >> synced DR filesystem, which is an HSM filesystem (as opposed to the >> production filesystem, which is disk only). Thus we'll be using GPFS to find >> candidate files (based on CTIME and/or MTIME) and passing the list to rsync >> to sync over. So we could pass the list to "touch" to touch the destination >> files before syncing to them, to force rsync to do a copy. >> >> -- >> Orlando >> >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 01 October 2012 10:20 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> Whoop! Got it! >>>> >>>> Attached is the complete patch against the rsync-3.0.9 release bundle: >>>> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >>>> >>>> The patch was taken with: >>>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >>>> 3.0.9.patch >>>> >>>> To build once patched - do: >>>> ./configure --enable-acl-support --enable-xattr-support >>>> --enable-gpfs- support >>>> >>>> (make sure that libacl1-dev, or equivalent, is available). >>>> >>>> Seems to work as desired. >>>> >>>> In the end, I removed all the "if DEBUG_GTE" statements from the >>>> gpfs.c code to get it working (!). >>>> >>>> -- >>>> Orlando >>>> >>>> On 28/09/12 15:46, Jez Tucker wrote: >>>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr >>>>> issues all >>>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a >> punt. >>>>> Grabbing rsync latest git. >>>>> >>>>>> -----Original Message----- >>>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>>>> Sent: 28 September 2012 15:11 >>>>>> To: gpfsug-discuss at gpfsug.org >>>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>>>> >>>>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>>>> two filesystems, same machine. >>>>>>>>> >>>>>>>>> sounds very, very evil but have you tried robocopy from one >>>>>>>>> share to another? Not ideal. >>>>>>>> >>>>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>>>> >>>>>>> >>>>>>> When I have tried doing this from Windows file servers the results >>>>>>> where not nice as Windows has ACL's for things like administrators >>>>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>>>> server the resultant ACL's were still borked. >>>>>>> >>>>>>>> Still looking for something sane. >>>>>>>> >>>>>>> >>>>>>> Would AFM not do the trick? >>>>>>> >>>>>>>> Good to see I'm not just missing something blatantly obvious >> though! >>>>>>>> >>>>>>> >>>>>>> What's your coding like? Depending how often you are going to be >>>>>>> attempting this I would fix the IBM patches for rsync so they work >>>>>>> on a modern version of rsync. Patch does not look to complicated to >> me. >>>>>>> >>>>>> >>>>>> Beyond my abilities it seems. If anyone fancies having a friday >>>>>> afternoon coding challenge, here's the problem: >>>>>> >>>>>> I'm trying to apply this patch: >>>>>> >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> >>>>>> to rsync - let's say version 3.0.6: >>>>>> >>>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> >>>>>> So I do: >>>>>> mkdir -p src/rsync >>>>>> cd src/rsync >>>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> wget >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> tar -zxf rsync-3.0.6.tar.gz >>>>>> cd rsync-3.0.6 >>>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>>>> >>>>>> ./configure --prefix=/tmp >>>>>> make >>>>>> >>>>>> >>>>>> Error message is: >>>>>> >>>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>>>> from gpfs.c:20: >>>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>>>> parameter list [enabled by default] >>>>>> proto.h:111:28: warning: its scope is only this definition or >>>>>> declaration, which is probably not what you want [enabled by >>>>>> default] >>>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>>>> here >>>>>> gpfs.c: In function 'gpfs_free_sxp': >>>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_free_list': >>>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>>>> [-Wimplicit-function-declaration] >>>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c:162:17: note: each undeclared identifier is reported only >>>>>> once for each function it appears in >>>>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_find_attr': >>>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_cache_attr': >>>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_send_attr': >>>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_get_attr': >>>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_set_attr': >>>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>>>> make: *** [gpfs.o] Error 1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Dr Orlando Richards >>>>>> Information Services >>>>>> IT Infrastructure Division >>>>>> Unix Section >>>>>> Tel: 0131 650 4994 >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> gpfsug-discuss mailing list >>>>>> gpfsug-discuss at gpfsug.org >>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>>> >>>>> _______________________________________________ >>>>> gpfsug-discuss mailing list >>>>> gpfsug-discuss at gpfsug.org >>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 15 12:31:02 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 15 Oct 2012 11:31:02 +0000 Subject: [gpfsug-discuss] gpfs dmapiEventTimout Message-ID: <39571EA9316BE44899D59C7A640C13F5305D18D2@WARVWEXC1.uk.deluxe-eu.com> If you have this set, what do you have yours set to and why? --- Jez Tucker Senior Sysadmin Rushes -------------- next part -------------- An HTML attachment was scrubbed... URL: From orlando.richards at ed.ac.uk Mon Oct 1 10:20:23 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:20:23 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696057.3080005@ed.ac.uk> Whoop! Got it! Attached is the complete patch against the rsync-3.0.9 release bundle: http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz The patch was taken with: diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync-3.0.9.patch To build once patched - do: ./configure --enable-acl-support --enable-xattr-support --enable-gpfs-support (make sure that libacl1-dev, or equivalent, is available). Seems to work as desired. In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c code to get it working (!). -- Orlando On 28/09/12 15:46, Jez Tucker wrote: > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > Grabbing rsync latest git. > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 28 September 2012 15:11 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 28/09/12 12:30, Jonathan Buzzard wrote: >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>> On 28/09/12 11:28, Barry Evans wrote: >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>> two filesystems, same machine. >>>>> >>>>> sounds very, very evil but have you tried robocopy from one share to >>>>> another? Not ideal. >>>> >>>> I've considered it - but not yet desperate enough to try it :) >>>> >>> >>> When I have tried doing this from Windows file servers the results >>> where not nice as Windows has ACL's for things like administrators >>> etc. that robocopy attempted to copy over. It was also painfully slow. >>> I found xcopy was a lot faster, though as my source was a Windows >>> server the resultant ACL's were still borked. >>> >>>> Still looking for something sane. >>>> >>> >>> Would AFM not do the trick? >>> >>>> Good to see I'm not just missing something blatantly obvious though! >>>> >>> >>> What's your coding like? Depending how often you are going to be >>> attempting this I would fix the IBM patches for rsync so they work on >>> a modern version of rsync. Patch does not look to complicated to me. >>> >> >> Beyond my abilities it seems. If anyone fancies having a friday afternoon >> coding challenge, here's the problem: >> >> I'm trying to apply this patch: >> >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> >> to rsync - let's say version 3.0.6: >> >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> >> So I do: >> mkdir -p src/rsync >> cd src/rsync >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> wget >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> tar -zxf rsync-3.0.6.tar.gz >> cd rsync-3.0.6 >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >> >> ./configure --prefix=/tmp >> make >> >> >> Error message is: >> >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c gpfs.c -o >> gpfs.o In file included from rsync.h:920:0, >> from gpfs.c:20: >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside parameter >> list [enabled by default] >> proto.h:111:28: warning: its scope is only this definition or declaration, >> which is probably not what you want [enabled by default] >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was here >> gpfs.c: In function 'gpfs_free_sxp': >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_free_list': >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >> [-Wimplicit-function-declaration] >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c:162:17: note: each undeclared identifier is reported only once for >> each function it appears in >> gpfs.c: In function 'gpfs_receive_attr_int': >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_find_attr': >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_cache_attr': >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_send_attr': >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_get_attr': >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_set_attr': >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_attr_get_changed': >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >> make: *** [gpfs.o] Error 1 >> >> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -------------- next part -------------- A non-text attachment was scrubbed... Name: gpfs_support_rsync-3.0.9.patch Type: text/x-patch Size: 33668 bytes Desc: not available URL: From Jez.Tucker at rushes.co.uk Mon Oct 1 10:33:42 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:33:42 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696057.3080005@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Nice. Can you put it on the github? https://github.com/gpfsug/gpfsug-tools > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:20 > To: gpfsug-discuss at gpfsug.org > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > Whoop! Got it! > > Attached is the complete patch against the rsync-3.0.9 release bundle: > http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > > The patch was taken with: > diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > 3.0.9.patch > > To build once patched - do: > ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- > support > > (make sure that libacl1-dev, or equivalent, is available). > > Seems to work as desired. > > In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c > code to get it working (!). > > -- > Orlando > > On 28/09/12 15:46, Jez Tucker wrote: > > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all > the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > > Grabbing rsync latest git. > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 28 September 2012 15:11 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>> two filesystems, same machine. > >>>>> > >>>>> sounds very, very evil but have you tried robocopy from one share > >>>>> to another? Not ideal. > >>>> > >>>> I've considered it - but not yet desperate enough to try it :) > >>>> > >>> > >>> When I have tried doing this from Windows file servers the results > >>> where not nice as Windows has ACL's for things like administrators > >>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>> I found xcopy was a lot faster, though as my source was a Windows > >>> server the resultant ACL's were still borked. > >>> > >>>> Still looking for something sane. > >>>> > >>> > >>> Would AFM not do the trick? > >>> > >>>> Good to see I'm not just missing something blatantly obvious though! > >>>> > >>> > >>> What's your coding like? Depending how often you are going to be > >>> attempting this I would fix the IBM patches for rsync so they work > >>> on a modern version of rsync. Patch does not look to complicated to me. > >>> > >> > >> Beyond my abilities it seems. If anyone fancies having a friday > >> afternoon coding challenge, here's the problem: > >> > >> I'm trying to apply this patch: > >> > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> > >> to rsync - let's say version 3.0.6: > >> > >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> > >> So I do: > >> mkdir -p src/rsync > >> cd src/rsync > >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> wget > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> tar -zxf rsync-3.0.6.tar.gz > >> cd rsync-3.0.6 > >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >> > >> ./configure --prefix=/tmp > >> make > >> > >> > >> Error message is: > >> > >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >> from gpfs.c:20: > >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >> parameter list [enabled by default] > >> proto.h:111:28: warning: its scope is only this definition or > >> declaration, which is probably not what you want [enabled by default] > >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >> here > >> gpfs.c: In function 'gpfs_free_sxp': > >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_free_list': > >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >> [-Wimplicit-function-declaration] > >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c:162:17: note: each undeclared identifier is reported only once > >> for each function it appears in > >> gpfs.c: In function 'gpfs_receive_attr_int': > >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_find_attr': > >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_cache_attr': > >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_send_attr': > >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_get_attr': > >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_set_attr': > >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_attr_get_changed': > >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >> make: *** [gpfs.o] Error 1 > >> > >> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > >> _______________________________________________ > >> gpfsug-discuss mailing list > >> gpfsug-discuss at gpfsug.org > >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. From orlando.richards at ed.ac.uk Mon Oct 1 10:41:55 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:41:55 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696563.3000507@ed.ac.uk> On 01/10/12 10:33, Jez Tucker wrote: > Nice. Can you put it on the github? > > https://github.com/gpfsug/gpfsug-tools > Sure - the whole rsync source, or just the patch? Also - I've discovered that it won't identify differences between ACLs to trigger a copy. When it does copy, it copies the ACLs though. For background - we're thinking of using this for building an asynchronously-synced DR filesystem, which is an HSM filesystem (as opposed to the production filesystem, which is disk only). Thus we'll be using GPFS to find candidate files (based on CTIME and/or MTIME) and passing the list to rsync to sync over. So we could pass the list to "touch" to touch the destination files before syncing to them, to force rsync to do a copy. -- Orlando > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:20 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> Whoop! Got it! >> >> Attached is the complete patch against the rsync-3.0.9 release bundle: >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >> >> The patch was taken with: >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >> 3.0.9.patch >> >> To build once patched - do: >> ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- >> support >> >> (make sure that libacl1-dev, or equivalent, is available). >> >> Seems to work as desired. >> >> In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c >> code to get it working (!). >> >> -- >> Orlando >> >> On 28/09/12 15:46, Jez Tucker wrote: >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. >>> Grabbing rsync latest git. >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 28 September 2012 15:11 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>> two filesystems, same machine. >>>>>>> >>>>>>> sounds very, very evil but have you tried robocopy from one share >>>>>>> to another? Not ideal. >>>>>> >>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>> >>>>> >>>>> When I have tried doing this from Windows file servers the results >>>>> where not nice as Windows has ACL's for things like administrators >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>> server the resultant ACL's were still borked. >>>>> >>>>>> Still looking for something sane. >>>>>> >>>>> >>>>> Would AFM not do the trick? >>>>> >>>>>> Good to see I'm not just missing something blatantly obvious though! >>>>>> >>>>> >>>>> What's your coding like? Depending how often you are going to be >>>>> attempting this I would fix the IBM patches for rsync so they work >>>>> on a modern version of rsync. Patch does not look to complicated to me. >>>>> >>>> >>>> Beyond my abilities it seems. If anyone fancies having a friday >>>> afternoon coding challenge, here's the problem: >>>> >>>> I'm trying to apply this patch: >>>> >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> >>>> to rsync - let's say version 3.0.6: >>>> >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> >>>> So I do: >>>> mkdir -p src/rsync >>>> cd src/rsync >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> wget >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> tar -zxf rsync-3.0.6.tar.gz >>>> cd rsync-3.0.6 >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>> >>>> ./configure --prefix=/tmp >>>> make >>>> >>>> >>>> Error message is: >>>> >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>> from gpfs.c:20: >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>> parameter list [enabled by default] >>>> proto.h:111:28: warning: its scope is only this definition or >>>> declaration, which is probably not what you want [enabled by default] >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>> here >>>> gpfs.c: In function 'gpfs_free_sxp': >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_free_list': >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>> [-Wimplicit-function-declaration] >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c:162:17: note: each undeclared identifier is reported only once >>>> for each function it appears in >>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_find_attr': >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_cache_attr': >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_send_attr': >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_get_attr': >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_set_attr': >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>> make: *** [gpfs.o] Error 1 >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> gpfsug-discuss mailing list >>>> gpfsug-discuss at gpfsug.org >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 1 10:47:13 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:47:13 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696563.3000507@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> I'd put the tarball, patch and binary. That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. Let me know your github user and I'll give you r/w. > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:42 > To: gpfsug main discussion list > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > On 01/10/12 10:33, Jez Tucker wrote: > > Nice. Can you put it on the github? > > > > https://github.com/gpfsug/gpfsug-tools > > > > Sure - the whole rsync source, or just the patch? > > Also - I've discovered that it won't identify differences between ACLs to > trigger a copy. When it does copy, it copies the ACLs though. > > For background - we're thinking of using this for building an asynchronously- > synced DR filesystem, which is an HSM filesystem (as opposed to the > production filesystem, which is disk only). Thus we'll be using GPFS to find > candidate files (based on CTIME and/or MTIME) and passing the list to rsync > to sync over. So we could pass the list to "touch" to touch the destination > files before syncing to them, to force rsync to do a copy. > > -- > Orlando > > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 01 October 2012 10:20 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> Whoop! Got it! > >> > >> Attached is the complete patch against the rsync-3.0.9 release bundle: > >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > >> > >> The patch was taken with: > >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > >> 3.0.9.patch > >> > >> To build once patched - do: > >> ./configure --enable-acl-support --enable-xattr-support > >> --enable-gpfs- support > >> > >> (make sure that libacl1-dev, or equivalent, is available). > >> > >> Seems to work as desired. > >> > >> In the end, I removed all the "if DEBUG_GTE" statements from the > >> gpfs.c code to get it working (!). > >> > >> -- > >> Orlando > >> > >> On 28/09/12 15:46, Jez Tucker wrote: > >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr > >>> issues all > >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a > punt. > >>> Grabbing rsync latest git. > >>> > >>>> -----Original Message----- > >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards > >>>> Sent: 28 September 2012 15:11 > >>>> To: gpfsug-discuss at gpfsug.org > >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >>>> > >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>>>> two filesystems, same machine. > >>>>>>> > >>>>>>> sounds very, very evil but have you tried robocopy from one > >>>>>>> share to another? Not ideal. > >>>>>> > >>>>>> I've considered it - but not yet desperate enough to try it :) > >>>>>> > >>>>> > >>>>> When I have tried doing this from Windows file servers the results > >>>>> where not nice as Windows has ACL's for things like administrators > >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>>>> I found xcopy was a lot faster, though as my source was a Windows > >>>>> server the resultant ACL's were still borked. > >>>>> > >>>>>> Still looking for something sane. > >>>>>> > >>>>> > >>>>> Would AFM not do the trick? > >>>>> > >>>>>> Good to see I'm not just missing something blatantly obvious > though! > >>>>>> > >>>>> > >>>>> What's your coding like? Depending how often you are going to be > >>>>> attempting this I would fix the IBM patches for rsync so they work > >>>>> on a modern version of rsync. Patch does not look to complicated to > me. > >>>>> > >>>> > >>>> Beyond my abilities it seems. If anyone fancies having a friday > >>>> afternoon coding challenge, here's the problem: > >>>> > >>>> I'm trying to apply this patch: > >>>> > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> > >>>> to rsync - let's say version 3.0.6: > >>>> > >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> > >>>> So I do: > >>>> mkdir -p src/rsync > >>>> cd src/rsync > >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> wget > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> tar -zxf rsync-3.0.6.tar.gz > >>>> cd rsync-3.0.6 > >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >>>> > >>>> ./configure --prefix=/tmp > >>>> make > >>>> > >>>> > >>>> Error message is: > >>>> > >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >>>> from gpfs.c:20: > >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >>>> parameter list [enabled by default] > >>>> proto.h:111:28: warning: its scope is only this definition or > >>>> declaration, which is probably not what you want [enabled by > >>>> default] > >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >>>> here > >>>> gpfs.c: In function 'gpfs_free_sxp': > >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_free_list': > >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >>>> [-Wimplicit-function-declaration] > >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c:162:17: note: each undeclared identifier is reported only > >>>> once for each function it appears in > >>>> gpfs.c: In function 'gpfs_receive_attr_int': > >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_find_attr': > >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_cache_attr': > >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_send_attr': > >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_get_attr': > >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_set_attr': > >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_attr_get_changed': > >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >>>> make: *** [gpfs.o] Error 1 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> -- > >>>> Dr Orlando Richards > >>>> Information Services > >>>> IT Infrastructure Division > >>>> Unix Section > >>>> Tel: 0131 650 4994 > >>>> > >>>> The University of Edinburgh is a charitable body, registered in > >>>> Scotland, with registration number SC005336. > >>>> _______________________________________________ > >>>> gpfsug-discuss mailing list > >>>> gpfsug-discuss at gpfsug.org > >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >>> > >>> _______________________________________________ > >>> gpfsug-discuss mailing list > >>> gpfsug-discuss at gpfsug.org > >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss From orlando.richards at ed.ac.uk Mon Oct 1 11:07:56 2012 From: orlando.richards at ed.ac.uk (orlando.richards at ed.ac.uk) Date: Mon, 1 Oct 2012 11:07:56 +0100 (BST) Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> Message-ID: On Mon, 1 Oct 2012, Jez Tucker wrote: > I'd put the tarball, patch and binary. > That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. > > Let me know your github user and I'll give you r/w. OrlandoRichards (orlando.richards at ed.ac.uk) > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:42 >> To: gpfsug main discussion list >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 01/10/12 10:33, Jez Tucker wrote: >>> Nice. Can you put it on the github? >>> >>> https://github.com/gpfsug/gpfsug-tools >>> >> >> Sure - the whole rsync source, or just the patch? >> >> Also - I've discovered that it won't identify differences between ACLs to >> trigger a copy. When it does copy, it copies the ACLs though. >> >> For background - we're thinking of using this for building an asynchronously- >> synced DR filesystem, which is an HSM filesystem (as opposed to the >> production filesystem, which is disk only). Thus we'll be using GPFS to find >> candidate files (based on CTIME and/or MTIME) and passing the list to rsync >> to sync over. So we could pass the list to "touch" to touch the destination >> files before syncing to them, to force rsync to do a copy. >> >> -- >> Orlando >> >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 01 October 2012 10:20 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> Whoop! Got it! >>>> >>>> Attached is the complete patch against the rsync-3.0.9 release bundle: >>>> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >>>> >>>> The patch was taken with: >>>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >>>> 3.0.9.patch >>>> >>>> To build once patched - do: >>>> ./configure --enable-acl-support --enable-xattr-support >>>> --enable-gpfs- support >>>> >>>> (make sure that libacl1-dev, or equivalent, is available). >>>> >>>> Seems to work as desired. >>>> >>>> In the end, I removed all the "if DEBUG_GTE" statements from the >>>> gpfs.c code to get it working (!). >>>> >>>> -- >>>> Orlando >>>> >>>> On 28/09/12 15:46, Jez Tucker wrote: >>>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr >>>>> issues all >>>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a >> punt. >>>>> Grabbing rsync latest git. >>>>> >>>>>> -----Original Message----- >>>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>>>> Sent: 28 September 2012 15:11 >>>>>> To: gpfsug-discuss at gpfsug.org >>>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>>>> >>>>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>>>> two filesystems, same machine. >>>>>>>>> >>>>>>>>> sounds very, very evil but have you tried robocopy from one >>>>>>>>> share to another? Not ideal. >>>>>>>> >>>>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>>>> >>>>>>> >>>>>>> When I have tried doing this from Windows file servers the results >>>>>>> where not nice as Windows has ACL's for things like administrators >>>>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>>>> server the resultant ACL's were still borked. >>>>>>> >>>>>>>> Still looking for something sane. >>>>>>>> >>>>>>> >>>>>>> Would AFM not do the trick? >>>>>>> >>>>>>>> Good to see I'm not just missing something blatantly obvious >> though! >>>>>>>> >>>>>>> >>>>>>> What's your coding like? Depending how often you are going to be >>>>>>> attempting this I would fix the IBM patches for rsync so they work >>>>>>> on a modern version of rsync. Patch does not look to complicated to >> me. >>>>>>> >>>>>> >>>>>> Beyond my abilities it seems. If anyone fancies having a friday >>>>>> afternoon coding challenge, here's the problem: >>>>>> >>>>>> I'm trying to apply this patch: >>>>>> >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> >>>>>> to rsync - let's say version 3.0.6: >>>>>> >>>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> >>>>>> So I do: >>>>>> mkdir -p src/rsync >>>>>> cd src/rsync >>>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> wget >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> tar -zxf rsync-3.0.6.tar.gz >>>>>> cd rsync-3.0.6 >>>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>>>> >>>>>> ./configure --prefix=/tmp >>>>>> make >>>>>> >>>>>> >>>>>> Error message is: >>>>>> >>>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>>>> from gpfs.c:20: >>>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>>>> parameter list [enabled by default] >>>>>> proto.h:111:28: warning: its scope is only this definition or >>>>>> declaration, which is probably not what you want [enabled by >>>>>> default] >>>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>>>> here >>>>>> gpfs.c: In function 'gpfs_free_sxp': >>>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_free_list': >>>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>>>> [-Wimplicit-function-declaration] >>>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c:162:17: note: each undeclared identifier is reported only >>>>>> once for each function it appears in >>>>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_find_attr': >>>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_cache_attr': >>>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_send_attr': >>>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_get_attr': >>>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_set_attr': >>>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>>>> make: *** [gpfs.o] Error 1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Dr Orlando Richards >>>>>> Information Services >>>>>> IT Infrastructure Division >>>>>> Unix Section >>>>>> Tel: 0131 650 4994 >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> gpfsug-discuss mailing list >>>>>> gpfsug-discuss at gpfsug.org >>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>>> >>>>> _______________________________________________ >>>>> gpfsug-discuss mailing list >>>>> gpfsug-discuss at gpfsug.org >>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 15 12:31:02 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 15 Oct 2012 11:31:02 +0000 Subject: [gpfsug-discuss] gpfs dmapiEventTimout Message-ID: <39571EA9316BE44899D59C7A640C13F5305D18D2@WARVWEXC1.uk.deluxe-eu.com> If you have this set, what do you have yours set to and why? --- Jez Tucker Senior Sysadmin Rushes -------------- next part -------------- An HTML attachment was scrubbed... URL: From orlando.richards at ed.ac.uk Mon Oct 1 10:20:23 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:20:23 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696057.3080005@ed.ac.uk> Whoop! Got it! Attached is the complete patch against the rsync-3.0.9 release bundle: http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz The patch was taken with: diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync-3.0.9.patch To build once patched - do: ./configure --enable-acl-support --enable-xattr-support --enable-gpfs-support (make sure that libacl1-dev, or equivalent, is available). Seems to work as desired. In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c code to get it working (!). -- Orlando On 28/09/12 15:46, Jez Tucker wrote: > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > Grabbing rsync latest git. > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 28 September 2012 15:11 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 28/09/12 12:30, Jonathan Buzzard wrote: >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>> On 28/09/12 11:28, Barry Evans wrote: >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>> two filesystems, same machine. >>>>> >>>>> sounds very, very evil but have you tried robocopy from one share to >>>>> another? Not ideal. >>>> >>>> I've considered it - but not yet desperate enough to try it :) >>>> >>> >>> When I have tried doing this from Windows file servers the results >>> where not nice as Windows has ACL's for things like administrators >>> etc. that robocopy attempted to copy over. It was also painfully slow. >>> I found xcopy was a lot faster, though as my source was a Windows >>> server the resultant ACL's were still borked. >>> >>>> Still looking for something sane. >>>> >>> >>> Would AFM not do the trick? >>> >>>> Good to see I'm not just missing something blatantly obvious though! >>>> >>> >>> What's your coding like? Depending how often you are going to be >>> attempting this I would fix the IBM patches for rsync so they work on >>> a modern version of rsync. Patch does not look to complicated to me. >>> >> >> Beyond my abilities it seems. If anyone fancies having a friday afternoon >> coding challenge, here's the problem: >> >> I'm trying to apply this patch: >> >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> >> to rsync - let's say version 3.0.6: >> >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> >> So I do: >> mkdir -p src/rsync >> cd src/rsync >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> wget >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> tar -zxf rsync-3.0.6.tar.gz >> cd rsync-3.0.6 >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >> >> ./configure --prefix=/tmp >> make >> >> >> Error message is: >> >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c gpfs.c -o >> gpfs.o In file included from rsync.h:920:0, >> from gpfs.c:20: >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside parameter >> list [enabled by default] >> proto.h:111:28: warning: its scope is only this definition or declaration, >> which is probably not what you want [enabled by default] >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was here >> gpfs.c: In function 'gpfs_free_sxp': >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_free_list': >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >> [-Wimplicit-function-declaration] >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c:162:17: note: each undeclared identifier is reported only once for >> each function it appears in >> gpfs.c: In function 'gpfs_receive_attr_int': >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_find_attr': >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_cache_attr': >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_send_attr': >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_get_attr': >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_set_attr': >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_attr_get_changed': >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >> make: *** [gpfs.o] Error 1 >> >> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -------------- next part -------------- A non-text attachment was scrubbed... Name: gpfs_support_rsync-3.0.9.patch Type: text/x-patch Size: 33668 bytes Desc: not available URL: From Jez.Tucker at rushes.co.uk Mon Oct 1 10:33:42 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:33:42 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696057.3080005@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Nice. Can you put it on the github? https://github.com/gpfsug/gpfsug-tools > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:20 > To: gpfsug-discuss at gpfsug.org > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > Whoop! Got it! > > Attached is the complete patch against the rsync-3.0.9 release bundle: > http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > > The patch was taken with: > diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > 3.0.9.patch > > To build once patched - do: > ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- > support > > (make sure that libacl1-dev, or equivalent, is available). > > Seems to work as desired. > > In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c > code to get it working (!). > > -- > Orlando > > On 28/09/12 15:46, Jez Tucker wrote: > > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all > the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > > Grabbing rsync latest git. > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 28 September 2012 15:11 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>> two filesystems, same machine. > >>>>> > >>>>> sounds very, very evil but have you tried robocopy from one share > >>>>> to another? Not ideal. > >>>> > >>>> I've considered it - but not yet desperate enough to try it :) > >>>> > >>> > >>> When I have tried doing this from Windows file servers the results > >>> where not nice as Windows has ACL's for things like administrators > >>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>> I found xcopy was a lot faster, though as my source was a Windows > >>> server the resultant ACL's were still borked. > >>> > >>>> Still looking for something sane. > >>>> > >>> > >>> Would AFM not do the trick? > >>> > >>>> Good to see I'm not just missing something blatantly obvious though! > >>>> > >>> > >>> What's your coding like? Depending how often you are going to be > >>> attempting this I would fix the IBM patches for rsync so they work > >>> on a modern version of rsync. Patch does not look to complicated to me. > >>> > >> > >> Beyond my abilities it seems. If anyone fancies having a friday > >> afternoon coding challenge, here's the problem: > >> > >> I'm trying to apply this patch: > >> > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> > >> to rsync - let's say version 3.0.6: > >> > >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> > >> So I do: > >> mkdir -p src/rsync > >> cd src/rsync > >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> wget > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> tar -zxf rsync-3.0.6.tar.gz > >> cd rsync-3.0.6 > >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >> > >> ./configure --prefix=/tmp > >> make > >> > >> > >> Error message is: > >> > >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >> from gpfs.c:20: > >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >> parameter list [enabled by default] > >> proto.h:111:28: warning: its scope is only this definition or > >> declaration, which is probably not what you want [enabled by default] > >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >> here > >> gpfs.c: In function 'gpfs_free_sxp': > >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_free_list': > >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >> [-Wimplicit-function-declaration] > >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c:162:17: note: each undeclared identifier is reported only once > >> for each function it appears in > >> gpfs.c: In function 'gpfs_receive_attr_int': > >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_find_attr': > >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_cache_attr': > >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_send_attr': > >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_get_attr': > >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_set_attr': > >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_attr_get_changed': > >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >> make: *** [gpfs.o] Error 1 > >> > >> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > >> _______________________________________________ > >> gpfsug-discuss mailing list > >> gpfsug-discuss at gpfsug.org > >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. From orlando.richards at ed.ac.uk Mon Oct 1 10:41:55 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:41:55 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696563.3000507@ed.ac.uk> On 01/10/12 10:33, Jez Tucker wrote: > Nice. Can you put it on the github? > > https://github.com/gpfsug/gpfsug-tools > Sure - the whole rsync source, or just the patch? Also - I've discovered that it won't identify differences between ACLs to trigger a copy. When it does copy, it copies the ACLs though. For background - we're thinking of using this for building an asynchronously-synced DR filesystem, which is an HSM filesystem (as opposed to the production filesystem, which is disk only). Thus we'll be using GPFS to find candidate files (based on CTIME and/or MTIME) and passing the list to rsync to sync over. So we could pass the list to "touch" to touch the destination files before syncing to them, to force rsync to do a copy. -- Orlando > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:20 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> Whoop! Got it! >> >> Attached is the complete patch against the rsync-3.0.9 release bundle: >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >> >> The patch was taken with: >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >> 3.0.9.patch >> >> To build once patched - do: >> ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- >> support >> >> (make sure that libacl1-dev, or equivalent, is available). >> >> Seems to work as desired. >> >> In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c >> code to get it working (!). >> >> -- >> Orlando >> >> On 28/09/12 15:46, Jez Tucker wrote: >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. >>> Grabbing rsync latest git. >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 28 September 2012 15:11 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>> two filesystems, same machine. >>>>>>> >>>>>>> sounds very, very evil but have you tried robocopy from one share >>>>>>> to another? Not ideal. >>>>>> >>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>> >>>>> >>>>> When I have tried doing this from Windows file servers the results >>>>> where not nice as Windows has ACL's for things like administrators >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>> server the resultant ACL's were still borked. >>>>> >>>>>> Still looking for something sane. >>>>>> >>>>> >>>>> Would AFM not do the trick? >>>>> >>>>>> Good to see I'm not just missing something blatantly obvious though! >>>>>> >>>>> >>>>> What's your coding like? Depending how often you are going to be >>>>> attempting this I would fix the IBM patches for rsync so they work >>>>> on a modern version of rsync. Patch does not look to complicated to me. >>>>> >>>> >>>> Beyond my abilities it seems. If anyone fancies having a friday >>>> afternoon coding challenge, here's the problem: >>>> >>>> I'm trying to apply this patch: >>>> >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> >>>> to rsync - let's say version 3.0.6: >>>> >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> >>>> So I do: >>>> mkdir -p src/rsync >>>> cd src/rsync >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> wget >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> tar -zxf rsync-3.0.6.tar.gz >>>> cd rsync-3.0.6 >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>> >>>> ./configure --prefix=/tmp >>>> make >>>> >>>> >>>> Error message is: >>>> >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>> from gpfs.c:20: >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>> parameter list [enabled by default] >>>> proto.h:111:28: warning: its scope is only this definition or >>>> declaration, which is probably not what you want [enabled by default] >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>> here >>>> gpfs.c: In function 'gpfs_free_sxp': >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_free_list': >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>> [-Wimplicit-function-declaration] >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c:162:17: note: each undeclared identifier is reported only once >>>> for each function it appears in >>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_find_attr': >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_cache_attr': >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_send_attr': >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_get_attr': >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_set_attr': >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>> make: *** [gpfs.o] Error 1 >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> gpfsug-discuss mailing list >>>> gpfsug-discuss at gpfsug.org >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 1 10:47:13 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:47:13 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696563.3000507@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> I'd put the tarball, patch and binary. That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. Let me know your github user and I'll give you r/w. > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:42 > To: gpfsug main discussion list > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > On 01/10/12 10:33, Jez Tucker wrote: > > Nice. Can you put it on the github? > > > > https://github.com/gpfsug/gpfsug-tools > > > > Sure - the whole rsync source, or just the patch? > > Also - I've discovered that it won't identify differences between ACLs to > trigger a copy. When it does copy, it copies the ACLs though. > > For background - we're thinking of using this for building an asynchronously- > synced DR filesystem, which is an HSM filesystem (as opposed to the > production filesystem, which is disk only). Thus we'll be using GPFS to find > candidate files (based on CTIME and/or MTIME) and passing the list to rsync > to sync over. So we could pass the list to "touch" to touch the destination > files before syncing to them, to force rsync to do a copy. > > -- > Orlando > > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 01 October 2012 10:20 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> Whoop! Got it! > >> > >> Attached is the complete patch against the rsync-3.0.9 release bundle: > >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > >> > >> The patch was taken with: > >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > >> 3.0.9.patch > >> > >> To build once patched - do: > >> ./configure --enable-acl-support --enable-xattr-support > >> --enable-gpfs- support > >> > >> (make sure that libacl1-dev, or equivalent, is available). > >> > >> Seems to work as desired. > >> > >> In the end, I removed all the "if DEBUG_GTE" statements from the > >> gpfs.c code to get it working (!). > >> > >> -- > >> Orlando > >> > >> On 28/09/12 15:46, Jez Tucker wrote: > >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr > >>> issues all > >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a > punt. > >>> Grabbing rsync latest git. > >>> > >>>> -----Original Message----- > >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards > >>>> Sent: 28 September 2012 15:11 > >>>> To: gpfsug-discuss at gpfsug.org > >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >>>> > >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>>>> two filesystems, same machine. > >>>>>>> > >>>>>>> sounds very, very evil but have you tried robocopy from one > >>>>>>> share to another? Not ideal. > >>>>>> > >>>>>> I've considered it - but not yet desperate enough to try it :) > >>>>>> > >>>>> > >>>>> When I have tried doing this from Windows file servers the results > >>>>> where not nice as Windows has ACL's for things like administrators > >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>>>> I found xcopy was a lot faster, though as my source was a Windows > >>>>> server the resultant ACL's were still borked. > >>>>> > >>>>>> Still looking for something sane. > >>>>>> > >>>>> > >>>>> Would AFM not do the trick? > >>>>> > >>>>>> Good to see I'm not just missing something blatantly obvious > though! > >>>>>> > >>>>> > >>>>> What's your coding like? Depending how often you are going to be > >>>>> attempting this I would fix the IBM patches for rsync so they work > >>>>> on a modern version of rsync. Patch does not look to complicated to > me. > >>>>> > >>>> > >>>> Beyond my abilities it seems. If anyone fancies having a friday > >>>> afternoon coding challenge, here's the problem: > >>>> > >>>> I'm trying to apply this patch: > >>>> > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> > >>>> to rsync - let's say version 3.0.6: > >>>> > >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> > >>>> So I do: > >>>> mkdir -p src/rsync > >>>> cd src/rsync > >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> wget > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> tar -zxf rsync-3.0.6.tar.gz > >>>> cd rsync-3.0.6 > >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >>>> > >>>> ./configure --prefix=/tmp > >>>> make > >>>> > >>>> > >>>> Error message is: > >>>> > >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >>>> from gpfs.c:20: > >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >>>> parameter list [enabled by default] > >>>> proto.h:111:28: warning: its scope is only this definition or > >>>> declaration, which is probably not what you want [enabled by > >>>> default] > >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >>>> here > >>>> gpfs.c: In function 'gpfs_free_sxp': > >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_free_list': > >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >>>> [-Wimplicit-function-declaration] > >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c:162:17: note: each undeclared identifier is reported only > >>>> once for each function it appears in > >>>> gpfs.c: In function 'gpfs_receive_attr_int': > >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_find_attr': > >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_cache_attr': > >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_send_attr': > >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_get_attr': > >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_set_attr': > >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_attr_get_changed': > >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >>>> make: *** [gpfs.o] Error 1 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> -- > >>>> Dr Orlando Richards > >>>> Information Services > >>>> IT Infrastructure Division > >>>> Unix Section > >>>> Tel: 0131 650 4994 > >>>> > >>>> The University of Edinburgh is a charitable body, registered in > >>>> Scotland, with registration number SC005336. > >>>> _______________________________________________ > >>>> gpfsug-discuss mailing list > >>>> gpfsug-discuss at gpfsug.org > >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >>> > >>> _______________________________________________ > >>> gpfsug-discuss mailing list > >>> gpfsug-discuss at gpfsug.org > >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss From orlando.richards at ed.ac.uk Mon Oct 1 11:07:56 2012 From: orlando.richards at ed.ac.uk (orlando.richards at ed.ac.uk) Date: Mon, 1 Oct 2012 11:07:56 +0100 (BST) Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> Message-ID: On Mon, 1 Oct 2012, Jez Tucker wrote: > I'd put the tarball, patch and binary. > That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. > > Let me know your github user and I'll give you r/w. OrlandoRichards (orlando.richards at ed.ac.uk) > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:42 >> To: gpfsug main discussion list >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 01/10/12 10:33, Jez Tucker wrote: >>> Nice. Can you put it on the github? >>> >>> https://github.com/gpfsug/gpfsug-tools >>> >> >> Sure - the whole rsync source, or just the patch? >> >> Also - I've discovered that it won't identify differences between ACLs to >> trigger a copy. When it does copy, it copies the ACLs though. >> >> For background - we're thinking of using this for building an asynchronously- >> synced DR filesystem, which is an HSM filesystem (as opposed to the >> production filesystem, which is disk only). Thus we'll be using GPFS to find >> candidate files (based on CTIME and/or MTIME) and passing the list to rsync >> to sync over. So we could pass the list to "touch" to touch the destination >> files before syncing to them, to force rsync to do a copy. >> >> -- >> Orlando >> >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 01 October 2012 10:20 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> Whoop! Got it! >>>> >>>> Attached is the complete patch against the rsync-3.0.9 release bundle: >>>> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >>>> >>>> The patch was taken with: >>>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >>>> 3.0.9.patch >>>> >>>> To build once patched - do: >>>> ./configure --enable-acl-support --enable-xattr-support >>>> --enable-gpfs- support >>>> >>>> (make sure that libacl1-dev, or equivalent, is available). >>>> >>>> Seems to work as desired. >>>> >>>> In the end, I removed all the "if DEBUG_GTE" statements from the >>>> gpfs.c code to get it working (!). >>>> >>>> -- >>>> Orlando >>>> >>>> On 28/09/12 15:46, Jez Tucker wrote: >>>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr >>>>> issues all >>>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a >> punt. >>>>> Grabbing rsync latest git. >>>>> >>>>>> -----Original Message----- >>>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>>>> Sent: 28 September 2012 15:11 >>>>>> To: gpfsug-discuss at gpfsug.org >>>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>>>> >>>>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>>>> two filesystems, same machine. >>>>>>>>> >>>>>>>>> sounds very, very evil but have you tried robocopy from one >>>>>>>>> share to another? Not ideal. >>>>>>>> >>>>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>>>> >>>>>>> >>>>>>> When I have tried doing this from Windows file servers the results >>>>>>> where not nice as Windows has ACL's for things like administrators >>>>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>>>> server the resultant ACL's were still borked. >>>>>>> >>>>>>>> Still looking for something sane. >>>>>>>> >>>>>>> >>>>>>> Would AFM not do the trick? >>>>>>> >>>>>>>> Good to see I'm not just missing something blatantly obvious >> though! >>>>>>>> >>>>>>> >>>>>>> What's your coding like? Depending how often you are going to be >>>>>>> attempting this I would fix the IBM patches for rsync so they work >>>>>>> on a modern version of rsync. Patch does not look to complicated to >> me. >>>>>>> >>>>>> >>>>>> Beyond my abilities it seems. If anyone fancies having a friday >>>>>> afternoon coding challenge, here's the problem: >>>>>> >>>>>> I'm trying to apply this patch: >>>>>> >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> >>>>>> to rsync - let's say version 3.0.6: >>>>>> >>>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> >>>>>> So I do: >>>>>> mkdir -p src/rsync >>>>>> cd src/rsync >>>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> wget >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> tar -zxf rsync-3.0.6.tar.gz >>>>>> cd rsync-3.0.6 >>>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>>>> >>>>>> ./configure --prefix=/tmp >>>>>> make >>>>>> >>>>>> >>>>>> Error message is: >>>>>> >>>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>>>> from gpfs.c:20: >>>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>>>> parameter list [enabled by default] >>>>>> proto.h:111:28: warning: its scope is only this definition or >>>>>> declaration, which is probably not what you want [enabled by >>>>>> default] >>>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>>>> here >>>>>> gpfs.c: In function 'gpfs_free_sxp': >>>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_free_list': >>>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>>>> [-Wimplicit-function-declaration] >>>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c:162:17: note: each undeclared identifier is reported only >>>>>> once for each function it appears in >>>>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_find_attr': >>>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_cache_attr': >>>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_send_attr': >>>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_get_attr': >>>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_set_attr': >>>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>>>> make: *** [gpfs.o] Error 1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Dr Orlando Richards >>>>>> Information Services >>>>>> IT Infrastructure Division >>>>>> Unix Section >>>>>> Tel: 0131 650 4994 >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> gpfsug-discuss mailing list >>>>>> gpfsug-discuss at gpfsug.org >>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>>> >>>>> _______________________________________________ >>>>> gpfsug-discuss mailing list >>>>> gpfsug-discuss at gpfsug.org >>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 15 12:31:02 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 15 Oct 2012 11:31:02 +0000 Subject: [gpfsug-discuss] gpfs dmapiEventTimout Message-ID: <39571EA9316BE44899D59C7A640C13F5305D18D2@WARVWEXC1.uk.deluxe-eu.com> If you have this set, what do you have yours set to and why? --- Jez Tucker Senior Sysadmin Rushes -------------- next part -------------- An HTML attachment was scrubbed... URL: From orlando.richards at ed.ac.uk Mon Oct 1 10:20:23 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:20:23 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696057.3080005@ed.ac.uk> Whoop! Got it! Attached is the complete patch against the rsync-3.0.9 release bundle: http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz The patch was taken with: diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync-3.0.9.patch To build once patched - do: ./configure --enable-acl-support --enable-xattr-support --enable-gpfs-support (make sure that libacl1-dev, or equivalent, is available). Seems to work as desired. In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c code to get it working (!). -- Orlando On 28/09/12 15:46, Jez Tucker wrote: > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > Grabbing rsync latest git. > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 28 September 2012 15:11 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 28/09/12 12:30, Jonathan Buzzard wrote: >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>> On 28/09/12 11:28, Barry Evans wrote: >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>> two filesystems, same machine. >>>>> >>>>> sounds very, very evil but have you tried robocopy from one share to >>>>> another? Not ideal. >>>> >>>> I've considered it - but not yet desperate enough to try it :) >>>> >>> >>> When I have tried doing this from Windows file servers the results >>> where not nice as Windows has ACL's for things like administrators >>> etc. that robocopy attempted to copy over. It was also painfully slow. >>> I found xcopy was a lot faster, though as my source was a Windows >>> server the resultant ACL's were still borked. >>> >>>> Still looking for something sane. >>>> >>> >>> Would AFM not do the trick? >>> >>>> Good to see I'm not just missing something blatantly obvious though! >>>> >>> >>> What's your coding like? Depending how often you are going to be >>> attempting this I would fix the IBM patches for rsync so they work on >>> a modern version of rsync. Patch does not look to complicated to me. >>> >> >> Beyond my abilities it seems. If anyone fancies having a friday afternoon >> coding challenge, here's the problem: >> >> I'm trying to apply this patch: >> >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> >> to rsync - let's say version 3.0.6: >> >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> >> So I do: >> mkdir -p src/rsync >> cd src/rsync >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >> wget >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rsync >> _gpfs_attrs-git.obj >> tar -zxf rsync-3.0.6.tar.gz >> cd rsync-3.0.6 >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >> >> ./configure --prefix=/tmp >> make >> >> >> Error message is: >> >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c gpfs.c -o >> gpfs.o In file included from rsync.h:920:0, >> from gpfs.c:20: >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside parameter >> list [enabled by default] >> proto.h:111:28: warning: its scope is only this definition or declaration, >> which is probably not what you want [enabled by default] >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was here >> gpfs.c: In function 'gpfs_free_sxp': >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_free_list': >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >> [-Wimplicit-function-declaration] >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c:162:17: note: each undeclared identifier is reported only once for >> each function it appears in >> gpfs.c: In function 'gpfs_receive_attr_int': >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_find_attr': >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_cache_attr': >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c: In function 'gpfs_send_attr': >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_get_attr': >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_set_attr': >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >> gpfs.c: In function 'gpfs_attr_get_changed': >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >> make: *** [gpfs.o] Error 1 >> >> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -------------- next part -------------- A non-text attachment was scrubbed... Name: gpfs_support_rsync-3.0.9.patch Type: text/x-patch Size: 33668 bytes Desc: not available URL: From Jez.Tucker at rushes.co.uk Mon Oct 1 10:33:42 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:33:42 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696057.3080005@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Nice. Can you put it on the github? https://github.com/gpfsug/gpfsug-tools > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:20 > To: gpfsug-discuss at gpfsug.org > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > Whoop! Got it! > > Attached is the complete patch against the rsync-3.0.9 release bundle: > http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > > The patch was taken with: > diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > 3.0.9.patch > > To build once patched - do: > ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- > support > > (make sure that libacl1-dev, or equivalent, is available). > > Seems to work as desired. > > In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c > code to get it working (!). > > -- > Orlando > > On 28/09/12 15:46, Jez Tucker wrote: > > Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all > the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. > > Grabbing rsync latest git. > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 28 September 2012 15:11 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>> two filesystems, same machine. > >>>>> > >>>>> sounds very, very evil but have you tried robocopy from one share > >>>>> to another? Not ideal. > >>>> > >>>> I've considered it - but not yet desperate enough to try it :) > >>>> > >>> > >>> When I have tried doing this from Windows file servers the results > >>> where not nice as Windows has ACL's for things like administrators > >>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>> I found xcopy was a lot faster, though as my source was a Windows > >>> server the resultant ACL's were still borked. > >>> > >>>> Still looking for something sane. > >>>> > >>> > >>> Would AFM not do the trick? > >>> > >>>> Good to see I'm not just missing something blatantly obvious though! > >>>> > >>> > >>> What's your coding like? Depending how often you are going to be > >>> attempting this I would fix the IBM patches for rsync so they work > >>> on a modern version of rsync. Patch does not look to complicated to me. > >>> > >> > >> Beyond my abilities it seems. If anyone fancies having a friday > >> afternoon coding challenge, here's the problem: > >> > >> I'm trying to apply this patch: > >> > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> > >> to rsync - let's say version 3.0.6: > >> > >> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> > >> So I do: > >> mkdir -p src/rsync > >> cd src/rsync > >> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >> wget > >> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs > >> ync > >> _gpfs_attrs-git.obj > >> tar -zxf rsync-3.0.6.tar.gz > >> cd rsync-3.0.6 > >> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >> > >> ./configure --prefix=/tmp > >> make > >> > >> > >> Error message is: > >> > >> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >> from gpfs.c:20: > >> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >> parameter list [enabled by default] > >> proto.h:111:28: warning: its scope is only this definition or > >> declaration, which is probably not what you want [enabled by default] > >> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >> here > >> gpfs.c: In function 'gpfs_free_sxp': > >> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_free_list': > >> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >> [-Wimplicit-function-declaration] > >> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c:162:17: note: each undeclared identifier is reported only once > >> for each function it appears in > >> gpfs.c: In function 'gpfs_receive_attr_int': > >> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_find_attr': > >> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_cache_attr': > >> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c: In function 'gpfs_send_attr': > >> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_get_attr': > >> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_set_attr': > >> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >> gpfs.c: In function 'gpfs_attr_get_changed': > >> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >> make: *** [gpfs.o] Error 1 > >> > >> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > >> _______________________________________________ > >> gpfsug-discuss mailing list > >> gpfsug-discuss at gpfsug.org > >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. From orlando.richards at ed.ac.uk Mon Oct 1 10:41:55 2012 From: orlando.richards at ed.ac.uk (Orlando Richards) Date: Mon, 01 Oct 2012 10:41:55 +0100 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> Message-ID: <50696563.3000507@ed.ac.uk> On 01/10/12 10:33, Jez Tucker wrote: > Nice. Can you put it on the github? > > https://github.com/gpfsug/gpfsug-tools > Sure - the whole rsync source, or just the patch? Also - I've discovered that it won't identify differences between ACLs to trigger a copy. When it does copy, it copies the ACLs though. For background - we're thinking of using this for building an asynchronously-synced DR filesystem, which is an HSM filesystem (as opposed to the production filesystem, which is disk only). Thus we'll be using GPFS to find candidate files (based on CTIME and/or MTIME) and passing the list to rsync to sync over. So we could pass the list to "touch" to touch the destination files before syncing to them, to force rsync to do a copy. -- Orlando > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:20 >> To: gpfsug-discuss at gpfsug.org >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> Whoop! Got it! >> >> Attached is the complete patch against the rsync-3.0.9 release bundle: >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >> >> The patch was taken with: >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >> 3.0.9.patch >> >> To build once patched - do: >> ./configure --enable-acl-support --enable-xattr-support --enable-gpfs- >> support >> >> (make sure that libacl1-dev, or equivalent, is available). >> >> Seems to work as desired. >> >> In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c >> code to get it working (!). >> >> -- >> Orlando >> >> On 28/09/12 15:46, Jez Tucker wrote: >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt. >>> Grabbing rsync latest git. >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 28 September 2012 15:11 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>> two filesystems, same machine. >>>>>>> >>>>>>> sounds very, very evil but have you tried robocopy from one share >>>>>>> to another? Not ideal. >>>>>> >>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>> >>>>> >>>>> When I have tried doing this from Windows file servers the results >>>>> where not nice as Windows has ACL's for things like administrators >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>> server the resultant ACL's were still borked. >>>>> >>>>>> Still looking for something sane. >>>>>> >>>>> >>>>> Would AFM not do the trick? >>>>> >>>>>> Good to see I'm not just missing something blatantly obvious though! >>>>>> >>>>> >>>>> What's your coding like? Depending how often you are going to be >>>>> attempting this I would fix the IBM patches for rsync so they work >>>>> on a modern version of rsync. Patch does not look to complicated to me. >>>>> >>>> >>>> Beyond my abilities it seems. If anyone fancies having a friday >>>> afternoon coding challenge, here's the problem: >>>> >>>> I'm trying to apply this patch: >>>> >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> >>>> to rsync - let's say version 3.0.6: >>>> >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> >>>> So I do: >>>> mkdir -p src/rsync >>>> cd src/rsync >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>> wget >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs >>>> ync >>>> _gpfs_attrs-git.obj >>>> tar -zxf rsync-3.0.6.tar.gz >>>> cd rsync-3.0.6 >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>> >>>> ./configure --prefix=/tmp >>>> make >>>> >>>> >>>> Error message is: >>>> >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>> from gpfs.c:20: >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>> parameter list [enabled by default] >>>> proto.h:111:28: warning: its scope is only this definition or >>>> declaration, which is probably not what you want [enabled by default] >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>> here >>>> gpfs.c: In function 'gpfs_free_sxp': >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_free_list': >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>> [-Wimplicit-function-declaration] >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c:162:17: note: each undeclared identifier is reported only once >>>> for each function it appears in >>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_find_attr': >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_cache_attr': >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c: In function 'gpfs_send_attr': >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_get_attr': >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_set_attr': >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>> make: *** [gpfs.o] Error 1 >>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> gpfsug-discuss mailing list >>>> gpfsug-discuss at gpfsug.org >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 1 10:47:13 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 1 Oct 2012 09:47:13 +0000 Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <50696563.3000507@ed.ac.uk> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> Message-ID: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> I'd put the tarball, patch and binary. That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. Let me know your github user and I'll give you r/w. > -----Original Message----- > From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > bounces at gpfsug.org] On Behalf Of Orlando Richards > Sent: 01 October 2012 10:42 > To: gpfsug main discussion list > Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > > On 01/10/12 10:33, Jez Tucker wrote: > > Nice. Can you put it on the github? > > > > https://github.com/gpfsug/gpfsug-tools > > > > Sure - the whole rsync source, or just the patch? > > Also - I've discovered that it won't identify differences between ACLs to > trigger a copy. When it does copy, it copies the ACLs though. > > For background - we're thinking of using this for building an asynchronously- > synced DR filesystem, which is an HSM filesystem (as opposed to the > production filesystem, which is disk only). Thus we'll be using GPFS to find > candidate files (based on CTIME and/or MTIME) and passing the list to rsync > to sync over. So we could pass the list to "touch" to touch the destination > files before syncing to them, to force rsync to do a copy. > > -- > Orlando > > > > >> -----Original Message----- > >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >> bounces at gpfsug.org] On Behalf Of Orlando Richards > >> Sent: 01 October 2012 10:20 > >> To: gpfsug-discuss at gpfsug.org > >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >> > >> Whoop! Got it! > >> > >> Attached is the complete patch against the rsync-3.0.9 release bundle: > >> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz > >> > >> The patch was taken with: > >> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- > >> 3.0.9.patch > >> > >> To build once patched - do: > >> ./configure --enable-acl-support --enable-xattr-support > >> --enable-gpfs- support > >> > >> (make sure that libacl1-dev, or equivalent, is available). > >> > >> Seems to work as desired. > >> > >> In the end, I removed all the "if DEBUG_GTE" statements from the > >> gpfs.c code to get it working (!). > >> > >> -- > >> Orlando > >> > >> On 28/09/12 15:46, Jez Tucker wrote: > >>> Well, since it's Friday and I've just been tracing gpfs_get_winattr > >>> issues all > >> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a > punt. > >>> Grabbing rsync latest git. > >>> > >>>> -----Original Message----- > >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- > >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards > >>>> Sent: 28 September 2012 15:11 > >>>> To: gpfsug-discuss at gpfsug.org > >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems > >>>> > >>>> On 28/09/12 12:30, Jonathan Buzzard wrote: > >>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: > >>>>>> On 28/09/12 11:28, Barry Evans wrote: > >>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. > >>>>>>> two filesystems, same machine. > >>>>>>> > >>>>>>> sounds very, very evil but have you tried robocopy from one > >>>>>>> share to another? Not ideal. > >>>>>> > >>>>>> I've considered it - but not yet desperate enough to try it :) > >>>>>> > >>>>> > >>>>> When I have tried doing this from Windows file servers the results > >>>>> where not nice as Windows has ACL's for things like administrators > >>>>> etc. that robocopy attempted to copy over. It was also painfully slow. > >>>>> I found xcopy was a lot faster, though as my source was a Windows > >>>>> server the resultant ACL's were still borked. > >>>>> > >>>>>> Still looking for something sane. > >>>>>> > >>>>> > >>>>> Would AFM not do the trick? > >>>>> > >>>>>> Good to see I'm not just missing something blatantly obvious > though! > >>>>>> > >>>>> > >>>>> What's your coding like? Depending how often you are going to be > >>>>> attempting this I would fix the IBM patches for rsync so they work > >>>>> on a modern version of rsync. Patch does not look to complicated to > me. > >>>>> > >>>> > >>>> Beyond my abilities it seems. If anyone fancies having a friday > >>>> afternoon coding challenge, here's the problem: > >>>> > >>>> I'm trying to apply this patch: > >>>> > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> > >>>> to rsync - let's say version 3.0.6: > >>>> > >>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> > >>>> So I do: > >>>> mkdir -p src/rsync > >>>> cd src/rsync > >>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz > >>>> wget > >>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ > >>>> rs > >>>> ync > >>>> _gpfs_attrs-git.obj > >>>> tar -zxf rsync-3.0.6.tar.gz > >>>> cd rsync-3.0.6 > >>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff > >>>> > >>>> ./configure --prefix=/tmp > >>>> make > >>>> > >>>> > >>>> Error message is: > >>>> > >>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c > >>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, > >>>> from gpfs.c:20: > >>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside > >>>> parameter list [enabled by default] > >>>> proto.h:111:28: warning: its scope is only this definition or > >>>> declaration, which is probably not what you want [enabled by > >>>> default] > >>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' > >>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was > >>>> here > >>>> gpfs.c: In function 'gpfs_free_sxp': > >>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_free_list': > >>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' > >>>> [-Wimplicit-function-declaration] > >>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c:162:17: note: each undeclared identifier is reported only > >>>> once for each function it appears in > >>>> gpfs.c: In function 'gpfs_receive_attr_int': > >>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_find_attr': > >>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_cache_attr': > >>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c: In function 'gpfs_send_attr': > >>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_get_attr': > >>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_set_attr': > >>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) > >>>> gpfs.c: In function 'gpfs_attr_get_changed': > >>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' > >>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) > >>>> make: *** [gpfs.o] Error 1 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> -- > >>>> Dr Orlando Richards > >>>> Information Services > >>>> IT Infrastructure Division > >>>> Unix Section > >>>> Tel: 0131 650 4994 > >>>> > >>>> The University of Edinburgh is a charitable body, registered in > >>>> Scotland, with registration number SC005336. > >>>> _______________________________________________ > >>>> gpfsug-discuss mailing list > >>>> gpfsug-discuss at gpfsug.org > >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >>> > >>> _______________________________________________ > >>> gpfsug-discuss mailing list > >>> gpfsug-discuss at gpfsug.org > >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > >>> > >> > >> > >> -- > >> -- > >> Dr Orlando Richards > >> Information Services > >> IT Infrastructure Division > >> Unix Section > >> Tel: 0131 650 4994 > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at gpfsug.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > -- > -- > Dr Orlando Richards > Information Services > IT Infrastructure Division > Unix Section > Tel: 0131 650 4994 > > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss From orlando.richards at ed.ac.uk Mon Oct 1 11:07:56 2012 From: orlando.richards at ed.ac.uk (orlando.richards at ed.ac.uk) Date: Mon, 1 Oct 2012 11:07:56 +0100 (BST) Subject: [gpfsug-discuss] Copying ACLs between filesystems In-Reply-To: <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> References: <506575DB.3010801@ed.ac.uk> <3316F8DC-2190-4560-846C-7DECCDB854EC@canditmedia.co.uk> <506584B3.8070404@ed.ac.uk> <1348831811.26504.12.camel@buzzard.phy.strath.ac.uk> <5065AFE7.4000008@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CD62D@WARVWEXC1.uk.deluxe-eu.com> <50696057.3080005@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDC87@WARVWEXC1.uk.deluxe-eu.com> <50696563.3000507@ed.ac.uk> <39571EA9316BE44899D59C7A640C13F5305CDCAE@WARVWEXC1.uk.deluxe-eu.com> Message-ID: On Mon, 1 Oct 2012, Jez Tucker wrote: > I'd put the tarball, patch and binary. > That way it'll all make sense in 3 years time and you won't have to go scrabbling for the original src. > > Let me know your github user and I'll give you r/w. OrlandoRichards (orlando.richards at ed.ac.uk) > >> -----Original Message----- >> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >> bounces at gpfsug.org] On Behalf Of Orlando Richards >> Sent: 01 October 2012 10:42 >> To: gpfsug main discussion list >> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >> >> On 01/10/12 10:33, Jez Tucker wrote: >>> Nice. Can you put it on the github? >>> >>> https://github.com/gpfsug/gpfsug-tools >>> >> >> Sure - the whole rsync source, or just the patch? >> >> Also - I've discovered that it won't identify differences between ACLs to >> trigger a copy. When it does copy, it copies the ACLs though. >> >> For background - we're thinking of using this for building an asynchronously- >> synced DR filesystem, which is an HSM filesystem (as opposed to the >> production filesystem, which is disk only). Thus we'll be using GPFS to find >> candidate files (based on CTIME and/or MTIME) and passing the list to rsync >> to sync over. So we could pass the list to "touch" to touch the destination >> files before syncing to them, to force rsync to do a copy. >> >> -- >> Orlando >> >>> >>>> -----Original Message----- >>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>> Sent: 01 October 2012 10:20 >>>> To: gpfsug-discuss at gpfsug.org >>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>> >>>> Whoop! Got it! >>>> >>>> Attached is the complete patch against the rsync-3.0.9 release bundle: >>>> http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz >>>> >>>> The patch was taken with: >>>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/ > gpfs_support_rsync- >>>> 3.0.9.patch >>>> >>>> To build once patched - do: >>>> ./configure --enable-acl-support --enable-xattr-support >>>> --enable-gpfs- support >>>> >>>> (make sure that libacl1-dev, or equivalent, is available). >>>> >>>> Seems to work as desired. >>>> >>>> In the end, I removed all the "if DEBUG_GTE" statements from the >>>> gpfs.c code to get it working (!). >>>> >>>> -- >>>> Orlando >>>> >>>> On 28/09/12 15:46, Jez Tucker wrote: >>>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr >>>>> issues all >>>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a >> punt. >>>>> Grabbing rsync latest git. >>>>> >>>>>> -----Original Message----- >>>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss- >>>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards >>>>>> Sent: 28 September 2012 15:11 >>>>>> To: gpfsug-discuss at gpfsug.org >>>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems >>>>>> >>>>>> On 28/09/12 12:30, Jonathan Buzzard wrote: >>>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote: >>>>>>>> On 28/09/12 11:28, Barry Evans wrote: >>>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either. >>>>>>>>> two filesystems, same machine. >>>>>>>>> >>>>>>>>> sounds very, very evil but have you tried robocopy from one >>>>>>>>> share to another? Not ideal. >>>>>>>> >>>>>>>> I've considered it - but not yet desperate enough to try it :) >>>>>>>> >>>>>>> >>>>>>> When I have tried doing this from Windows file servers the results >>>>>>> where not nice as Windows has ACL's for things like administrators >>>>>>> etc. that robocopy attempted to copy over. It was also painfully slow. >>>>>>> I found xcopy was a lot faster, though as my source was a Windows >>>>>>> server the resultant ACL's were still borked. >>>>>>> >>>>>>>> Still looking for something sane. >>>>>>>> >>>>>>> >>>>>>> Would AFM not do the trick? >>>>>>> >>>>>>>> Good to see I'm not just missing something blatantly obvious >> though! >>>>>>>> >>>>>>> >>>>>>> What's your coding like? Depending how often you are going to be >>>>>>> attempting this I would fix the IBM patches for rsync so they work >>>>>>> on a modern version of rsync. Patch does not look to complicated to >> me. >>>>>>> >>>>>> >>>>>> Beyond my abilities it seems. If anyone fancies having a friday >>>>>> afternoon coding challenge, here's the problem: >>>>>> >>>>>> I'm trying to apply this patch: >>>>>> >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> >>>>>> to rsync - let's say version 3.0.6: >>>>>> >>>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> >>>>>> So I do: >>>>>> mkdir -p src/rsync >>>>>> cd src/rsync >>>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz >>>>>> wget >>>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/ >>>>>> rs >>>>>> ync >>>>>> _gpfs_attrs-git.obj >>>>>> tar -zxf rsync-3.0.6.tar.gz >>>>>> cd rsync-3.0.6 >>>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff >>>>>> >>>>>> ./configure --prefix=/tmp >>>>>> make >>>>>> >>>>>> >>>>>> Error message is: >>>>>> >>>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c >>>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0, >>>>>> from gpfs.c:20: >>>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside >>>>>> parameter list [enabled by default] >>>>>> proto.h:111:28: warning: its scope is only this definition or >>>>>> declaration, which is probably not what you want [enabled by >>>>>> default] >>>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr' >>>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was >>>>>> here >>>>>> gpfs.c: In function 'gpfs_free_sxp': >>>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_free_list': >>>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE' >>>>>> [-Wimplicit-function-declaration] >>>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c:162:17: note: each undeclared identifier is reported only >>>>>> once for each function it appears in >>>>>> gpfs.c: In function 'gpfs_receive_attr_int': >>>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_find_attr': >>>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_cache_attr': >>>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c: In function 'gpfs_send_attr': >>>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_get_attr': >>>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_set_attr': >>>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function) >>>>>> gpfs.c: In function 'gpfs_attr_get_changed': >>>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr' >>>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function) >>>>>> make: *** [gpfs.o] Error 1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Dr Orlando Richards >>>>>> Information Services >>>>>> IT Infrastructure Division >>>>>> Unix Section >>>>>> Tel: 0131 650 4994 >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> gpfsug-discuss mailing list >>>>>> gpfsug-discuss at gpfsug.org >>>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>>> >>>>> _______________________________________________ >>>>> gpfsug-discuss mailing list >>>>> gpfsug-discuss at gpfsug.org >>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Dr Orlando Richards >>>> Information Services >>>> IT Infrastructure Division >>>> Unix Section >>>> Tel: 0131 650 4994 >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at gpfsug.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >> >> >> -- >> -- >> Dr Orlando Richards >> Information Services >> IT Infrastructure Division >> Unix Section >> Tel: 0131 650 4994 >> >> The University of Edinburgh is a charitable body, registered in Scotland, with >> registration number SC005336. >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From Jez.Tucker at rushes.co.uk Mon Oct 15 12:31:02 2012 From: Jez.Tucker at rushes.co.uk (Jez Tucker) Date: Mon, 15 Oct 2012 11:31:02 +0000 Subject: [gpfsug-discuss] gpfs dmapiEventTimout Message-ID: <39571EA9316BE44899D59C7A640C13F5305D18D2@WARVWEXC1.uk.deluxe-eu.com> If you have this set, what do you have yours set to and why? --- Jez Tucker Senior Sysadmin Rushes -------------- next part -------------- An HTML attachment was scrubbed... URL: