[gpfsug-discuss] Change uidNumber and gidNumber for billions of files

[Laurence Horrocks-Barlow]

I seem to remember Marc Kaplan discussing using the ILM and mmfind for this.

There is a presentation from 2018 which skims on an example

http://files.gpfsug.org/presentations/2018/USA/SpectrumScalePolicyBP.pdf

-- Lauz

On 10/06/2020 23:40, Kerner, Chad A wrote:
> You can do a policy scan though and get a list of files that have ACLs applied to them. Then you would not have to check every file with a shell utility or C, just process that list.  Likewise, you can get the uid/gid as well and process that list with the new mapping(split it into multiple lists, processing multiple threads on multiple machines).
>
> While it is by no means the prettiest or possibly best way to handle the POSIX ACLs, I had whipped up a python api for it: https://github.com/ckerner/ssacl . It only does POSIX though. We use it in conjunction with acls (https://github.com/ckerner/acls), an ls replacement that shows effective user/group permissions based off of the acl's because most often the user would just look at the POSIX perms and say something is broken, without checking the acl.
>
> --
> Chad Kerner, Senior Storage Engineer
> Storage Enabling Technologies
> National Center for Supercomputing Applications
> University of Illinois, Urbana-Champaign
>
> On 6/10/20, 5:30 PM, "gpfsug-discuss-bounces at spectrumscale.org on behalf of Jonathan Buzzard" <gpfsug-discuss-bounces at spectrumscale.org on behalf of jonathan.buzzard at strath.ac.uk> wrote:
>
>      On 10/06/2020 16:31, Lohit Valleru wrote:
>      
>      [SNIP]
>      
>       > I might mostly start small with a single lab, and only change files
>       > without ACLs. May I know if anyone has a method/tool to find out
>      which > files/dirs have NFS4 ACLs set? As far as we know - it is just one
>       > fileset/lab, but it would be good to confirm if we have them set
>       > across any other files/dirs in the filesystem. The usual methods do
>       > not seem to work.
>      
>      Use mmgetacl a file at a time and try and do something with the output?
>      
>      Tools to manipulate ACL's from on GPFS mounted nodes suck donkey balls,
>      and have been that way for over a decade. Last time I raised this with
>      IBM I was told that was by design...
>      
>      If they are CES then look at it client side from a Windows node?
>      
>      The alternative is to write something in C that calls gpfs_getacl.
>      
>      However it was an evening to get a basic UID remap code working in C. It
>      would not take much more effort to make it handle ACL's. As such I would
>      work on the premise that there are ACL's and handle it.
>      
>      JAB.
>      
>      --
>      Jonathan A. Buzzard                         Tel: +44141-5483420
>      HPC System Administrator, ARCHIE-WeSt.
>      University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
>      _______________________________________________
>      gpfsug-discuss mailing list
>      gpfsug-discuss at spectrumscale.org
>      http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>      
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss