[gpfsug-discuss] Mass UID migration suggestions

[Buterbaugh, Kevin L]

Hi Luke,

I’ve got an off the wall suggestion for you, which may or may not work depending on whether or not you have any UID conflicts with old and new UIDs … this won’t actually speed things up but it will eliminate the “downtime” for your users.  And the big caveat is that there can’t be any UID conflicts … i.e. someone’s new UID can’t be someone else’s old UID.

Given that … what if you set an ACL to allow access to both their old and new UIDs … then change their UID to the new UID … then chown the files to the new UID and remove the ACL?  More work for you, but no downtime for them.

We actually may need to do something similar as we will need to change Windows-assigned UID’s based on SIDs to “correct” UIDs at some point in the future on one of our storage systems.

If someone has a better way to solve your problem, I hope they’ll post it to the list, as it may help us as well.  HTHAL.  Thanks…

Kevin

On Jun 30, 2017, at 10:20 AM, hpc-luke at uconn.edu<mailto:hpc-luke at uconn.edu> wrote:

Hello,

We're trying to change most of our users uids, is there a clean way to
migrate all of one users files with say `mmapplypolicy`? We have to change the
owner of around 273539588 files, and my estimates for runtime are around 6 days.

What we've been doing is indexing all of the files and splitting them up by
owner which takes around an hour, and then we were locking the user out while we
chown their files. I made it multi threaded as it weirdly gave a 10% speedup
despite my expectation that multi threading access from a single node would not
give any speedup.

Generally I'm looking for advice on how to make the chowning faster. Would
spreading the chowning processes over multiple nodes improve performance? Should
I not stat the files before running lchown on them, since lchown checks the file
before changing it? I saw mention of inodescan(), in an old gpfsug email, which
speeds up disk read access, by not guaranteeing that the data is up to date. We
have a maintenance day coming up where all users will be locked out, so the file
handles(?) from GPFS's perspective will not be able to go stale. Is there a
function with similar constraints to inodescan that I can use to speed up this
process?

Thank you for your time,

Luke
Storrs-HPC
University of Connecticut
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170630/641d499f/attachment-0002.htm>