[gpfsug-discuss] Get list of filesets _without_runningmmlsfileset?

Thu Jan 10 20:42:50 GMT 2019

Hi Andrew / All,

Well, it does _sound_ useful, but in its current state it’s really not for several reasons, mainly having to do with it being coded in a moderately site-specific way.  It needs an overhaul anyway, so I’m going to look at getting rid of as much of that as possible (there’s some definite low-hanging fruit there) and, for the site-specific things that can’t be gotten rid of, maybe consolidating them into one place in the code so that the script could be more generally useful if you just change those values.

If I can accomplish those things, then yes, we’d be glad to share the script.

But I’ve also realized that I didn’t _entirely_ answer my original question.  Yes, mmlsquota will show me all the filesets … but I also need to know the junction path for each of those filesets.  One of the main reasons we wrote this script in the first place is that if you run mmlsquota you see that you have no limits on about 60 filesets (currently we use fileset quotas only on our filesets) … and that’s because there are no user (or group) quotas in those filesets.  The script, however, reads in that text file that is created nightly by root that is nothing more than the output of “mmlsfileset <filesystem name>”, gets the junction path, looks up the GID of the junction path, and sees if you’re a member of that group.  If you’re not, well, no sense in showing you anything about that fileset.  But, of course, if you are a member of that group, then we do want to show you the fileset quota for that fileset.

So … my question now is, “Is there a way for a non-root user” to get the junction path for the fileset(s)?  Thanks…

Kevin

—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633

On Jan 9, 2019, at 7:13 PM, Andrew Beattie <abeattie at au1.ibm.com<mailto:abeattie at au1.ibm.com>> wrote:

Kevin,

That sounds like a useful script
would you care to share?

Thanks
Andrew Beattie
Software Defined Storage  - IT Specialist
Phone: 614-2133-7927
E-mail: abeattie at au1.ibm.com<mailto:abeattie at au1.ibm.com>

----- Original message -----
From: "Buterbaugh, Kevin L" <Kevin.Buterbaugh at Vanderbilt.Edu<mailto:Kevin.Buterbaugh at Vanderbilt.Edu>>
Sent by: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Cc:
Subject: Re: [gpfsug-discuss] Get list of filesets _without_ running mmlsfileset?
Date: Thu, Jan 10, 2019 9:22 AM

Hi All,

Let me answer Skylar’s questions in another e-mail, which may also tell whether the rest API is a possibility or not.

The Python script in question is to display quota information for a user.  The mmlsquota command has a couple of issues:  1) its output is confusing to some of our users, 2) more significantly, it displays a ton of information that doesn’t apply to the user running it.  For example, it will display all the filesets in a filesystem whether or not the user has access to them.  So the Python script figures out what group(s) the user is a member of and only displays information pertinent to them (i.e. the group of the fileset junction path is a group this user is a member of) … and in a simplified (and potentially colorized) output format.

And typing that preceding paragraph caused the lightbulb to go off … I know the answer to my own question … have the script run mmlsquota and get the full list of filesets from that, then parse that to determine which ones I actually need to display quota information for.  Thanks!

Kevin
—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633

On Jan 9, 2019, at 4:42 PM, Simon Thompson <S.J.Thompson at bham.ac.uk<mailto:S.J.Thompson at bham.ac.uk>> wrote:

Hi Kevin,

Have you looked at the rest API?

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ibm.com%2Fsupport%2Fknowledgecenter%2Fen%2FSTXKQY_5.0.2%2Fcom.ibm.spectrum.scale.v5r02.doc%2Fbl1adm_listofapicommands.htm&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C36fb451ce9a945f5e0cb08d67683af85%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636826705300525885&sdata=uotWilntiZa2E9RIBE2ikhxxBm3Mk3y%2FW%2FKUHovaJpY%3D&reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ibm.com%2Fsupport%2Fknowledgecenter%2Fen%2FSTXKQY_5.0.2%2Fcom.ibm.spectrum.scale.v5r02.doc%2Fbl1adm_listofapicommands.htm&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cc1ffac821c5f4524104908d67698e948%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636826796466979672&sdata=0%2FsV4sgFJmyJjaETdNtmsRP21pm5zFSYdpr9eNtDSs4%3D&reserved=0>

I don't know how much access control there is available in the API so not sure if you could lock some sort of service user down to just the get filesets command?

Simon
_______________________________________
From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> [gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>] on behalf of Buterbaugh, Kevin L [Kevin.Buterbaugh at Vanderbilt.Edu<mailto:Kevin.Buterbaugh at Vanderbilt.Edu>]
Sent: 08 January 2019 22:12
To: gpfsug main discussion list
Subject: [gpfsug-discuss] Get list of filesets _without_ running mmlsfileset?

Hi All,

Happy New Year to all!  Personally, I’ll gladly and gratefully settle for 2019 not being a dumpster fire like 2018 was (those who attended my talk at the user group meeting at SC18 know what I’m referring to), but I certainly wish all of you the best!

Is there a way to get a list of the filesets in a filesystem without running mmlsfileset?  I was kind of expecting to find them in one of the config files somewhere under /var/mmfs but haven’t found them yet in the searching I’ve done.

The reason I’m asking is that we have a Python script that users can run that needs to get a list of all the filesets in a filesystem.  There are obviously multiple issues with that, so the workaround we’re using for now is to have a cron job which runs mmlsfileset once a day and dumps it out to a text file, which the script then reads.  That’s sub-optimal for any day on which a fileset gets created or deleted, so I’m looking for a better way … one which doesn’t require root privileges and preferably doesn’t involve running a GPFS command at all.

Thanks in advance.

Kevin

P.S.  I am still working on metadata and iSCSI testing and will report back on that when complete.
P.P.S.  We ended up adding our new NSDs comprised of (not really) 12 TB disks to the capacity pool and things are working fine.

—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu><mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C36fb451ce9a945f5e0cb08d67683af85%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636826705300525885&sdata=WSijRrjhOgQyuWsh9K8ckpjf%2F2CkXfZW1n%2BJw5Gw5tw%3D&reserved=0
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cc1ffac821c5f4524104908d67698e948%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636826796466989681&sdata=u32oTmCl3WXoCpPQQplOtAeAuzfNfHBjUmg%2BU194rbI%3D&reserved=0>

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cc1ffac821c5f4524104908d67698e948%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636826796467009700&sdata=Xfz4JiItI8ukHgnvO5YoN27jVpk6Ngsk03NtMrKJcHk%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190110/ac7ec2aa/attachment-0002.htm>