[gpfsug-discuss] Hardening sudo wrapper?

Wei Guo Wei1.Guo at UTSouthwestern.edu
Fri Feb 24 23:10:07 GMT 2017


As per the knowledge page suggested (https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.1/com.ibm.spectrum.scale.v4r21.doc/bl1adm_configsudo.htm), a sudo wapper can work around with PermitRootLogin no.

However, giving sudo right to a gpfsadmin account with /usr/bin/scp could be dangerous in the case of this gpfsadmin account been compromised. eg.

[gpfsadmin at adminNode ~] $ sudo /usr/bin/scp `/bin/echo /dev/random` /path/to/any_important_files.txt

Is it possible to remove scp from the sudoers commands?

Instead of the recommended here,
# Allow members of the gpfs group to run all commands but only selected commands without a password:
%gpfsadmin ALL=(ALL) PASSWD: ALL, NOPASSWD: /usr/lpp/mmfs/bin/mmremote, /usr/bin/scp, /bin/echo, /usr/lpp/mmfs/bin/mmsdrrestore

We would like to have this line like this:
 # Disabled command alias
Cmnd_alias MMDELCMDS = /usr/lpp/mmfs/bin/mmdeldisk, /usr/lpp/mmfs/bin/mmdelfileset, /usr/lpp/mmfs/bin/mmdelfs, /usr/lpp/mmfs/bin/mmdelnsd, /usr/lpp/mmfs/bin/mmdelsnapshot

%gpfsadmin ALL=(root : gpfsadmin) NOPASSWD: /bin/echo, /usr/lpp/mmfs/bin/​, !MMDELCMDS

In this case, we limit the gpfsadmin group user to run only selected mm commands, also not including /usr/bin/scp.
In the event of system breach, by loosing gpfsadmin group user account, scp will overwrite system config / user data.

From my initial test, this seems to be OK for basic admin commands (such as mmstartup, mmshutdown, mmrepquota, mmchfs),  but it did not pass the mmcommon test scpwrap command.
        ​[gpfsadmin at adminNode ~]$ sudo /usr/lpp/mmfs/bin/mmcommon test scpwrap node1
        sudo: no tty present and no askpass program specified
        lost connection
        mmcommon: Remote copy file command "/usr/lpp/mmfs/bin/scpwrap" failed (push operation).
                     Return code is 1.
        mmcommon test scpwrap: Command failed. Examine previous error messages to determine cause.
        [gpfsadmin at adminNode ~]$ sudo /usr/lpp/mmfs/bin/mmcommon test sshwrap node1
        mmcommon test sshwrap: Command successfully completed

It is unclear to me now that what exactly does the scp do in the sudo wrapper in the GPFS 4.2.0 version as per Yuri Volobuev's note GPFS and Remote Shell (https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20(GPFS)/page/GPFS%20and%20Remote%20Shell).
Will the mmsdrrestore still use scp or rcp to copy the cluster configuration file mmsdrfs around from the central node? Or it uses RPC to synchronize?
Are we OK to drop scp/rcp and limit the commands to run? Is there any risk, security wise and performance wise?
Can we limit the gpfsadmin account to a very very small level of privilege?
I have send this message to gpfs at us.ibm.com and posted at developer works, but I think the answer could benefit other users.

Thanks

Wei Guo

________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of gpfsug-discuss-request at spectrumscale.org <gpfsug-discuss-request at spectrumscale.org>
Sent: Friday, February 24, 2017 1:39 PM
To: gpfsug-discuss at spectrumscale.org
Subject: gpfsug-discuss Digest, Vol 61, Issue 46

Send gpfsug-discuss mailing list submissions to
        gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
        http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
        gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
        gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."


Today's Topics:

   1. NFS Permission matchup to mmnfs command (Shaun Anderson)
   2. Re: waiting for conn rdmas < conn maxrdmas (Aaron Knister)
   3. Re: waiting for conn rdmas < conn maxrdmas (Sven Oehme)


----------------------------------------------------------------------

Message: 1
Date: Fri, 24 Feb 2017 16:58:34 +0000
From: Shaun Anderson <SAnderson at convergeone.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] NFS Permission matchup to mmnfs command
Message-ID: <1487955513211.95497 at convergeone.com>
Content-Type: text/plain; charset="iso-8859-1"

I have a customer currently using native NFS and we are going to move them over the CES.  I'm looking at the mmnfs command and trying to map the nfs export arguments with the CES arguments. My customer has these currently:


no_wdelay,
nohide,
rw,
sync,
no_root_squash,
no_all_squash


I have this so far:

mmnfs export add /gpfs/ltfsee/<folder> --client XX.XX.XX.XX (

Access_Type=RW,

Squash=no_root_squash,noidsquash,

NFS_COMMIT=true

)

So the only arguments that don't appear accounted for is the 'nohide' parameter.

Does this look right?

SHAUN ANDERSON
STORAGE ARCHITECT
O 208.577.2112
M 214.263.7014


NOTICE:  This email message and any attachments here to may contain confidential
information.  Any unauthorized review, use, disclosure, or distribution of such
information is prohibited.  If you are not the intended recipient, please contact
the sender by reply email and destroy the original message and all copies of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20170224/afcd7b9a/attachment-0001.html>

------------------------------

Message: 2
Date: Fri, 24 Feb 2017 14:31:08 -0500
From: Aaron Knister <aaron.s.knister at nasa.gov>
To: <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] waiting for conn rdmas < conn maxrdmas
Message-ID: <ce8c2096-c56c-1df1-7c71-6a015df7015a at nasa.gov>
Content-Type: text/plain; charset="windows-1252"; format=flowed

Interesting, thanks Sven!

Could "resources" I'm running out of include NSD server queues?

On 2/23/17 12:12 PM, Sven Oehme wrote:
> all this waiter shows is that you have more in flight than the node or
> connection can currently serve. the reasons for that can be
> misconfiguration or you simply run out of resources on the node, not the
> connection. with latest code you shouldn't see this anymore for node
> limits as the system automatically adjusts the number of maximum RDMA's
> according to the systems Node capabilities :
>
> you should see messages in your mmfslog like :
>
> 2017-02-23_06:19:50.056-0800: [I] VERBS RDMA starting with
> verbsRdmaCm=no verbsRdmaSend=yes verbsRdmaUseMultiCqThreads=yes
> verbsRdmaUseCompVectors=yes
> 2017-02-23_06:19:50.078-0800: [I] VERBS RDMA library libibverbs.so
> (version >= 1.1) loaded and initialized.
> 2017-02-23_06:19:50.078-0800: [I] VERBS RDMA verbsRdmasPerNode increased
> from*_3072 to 3740 because verbsRdmasPerNodeOptimize is set to yes._*
> 2017-02-23_06:19:50.121-0800: [I] VERBS RDMA discover mlx5_5 port 1
> transport IB link  IB NUMA node 16 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB9CD state ACTIVE
> 2017-02-23_06:19:50.137-0800: [I] VERBS RDMA discover mlx5_4 port 1
> transport IB link  IB NUMA node 16 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB9CC state ACTIVE
> 2017-02-23_06:19:50.153-0800: [I] VERBS RDMA discover mlx5_3 port 1
> transport IB link  IB NUMA node  1 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB751 state ACTIVE
> 2017-02-23_06:19:50.169-0800: [I] VERBS RDMA discover mlx5_2 port 1
> transport IB link  IB NUMA node  1 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB750 state ACTIVE
> 2017-02-23_06:19:50.185-0800: [I] VERBS RDMA discover mlx5_1 port 1
> transport IB link  IB NUMA node  0 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB78D state ACTIVE
> 2017-02-23_06:19:50.201-0800: [I] VERBS RDMA discover mlx5_0 port 1
> transport IB link  IB NUMA node  0 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB78C state ACTIVE
>
> we want to eliminate all this configurable limits eventually, but this
> takes time, but as you can see above, we make progress on each release  :-)
>
> Sven
>
>
>
>
> On Thu, Feb 23, 2017 at 9:05 AM Aaron Knister <aaron.s.knister at nasa.gov
> <mailto:aaron.s.knister at nasa.gov>> wrote:
>
>     On a particularly heavy loaded NSD server I'm seeing a lot of these
>     messages:
>
>     0x7FFFF08B63E0 (  15539) waiting 0.004139456 seconds, NSDThread: on
>     ThCond 0x7FFFA80772C8 (0x7FFFA80772C8) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF08EED80 (  15584) waiting 0.004075718 seconds, NSDThread: on
>     ThCond 0x7FFF680008F8 (0x7FFF680008F8) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF08FDF00 (  15596) waiting 0.003965504 seconds, NSDThread: on
>     ThCond 0x7FFF8C00E288 (0x7FFF8C00E288) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF09185A0 (  15617) waiting 0.003916346 seconds, NSDThread: on
>     ThCond 0x7FFF9000CB18 (0x7FFF9000CB18) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF092B380 (  15632) waiting 0.003659610 seconds, NSDThread: on
>     ThCond 0x1DB04B8 (0x1DB04B8) (VERBSEventWaitCondvar), reason 'waiting
>     for conn rdmas < conn maxrdmas'
>
>     I've tried tweaking verbsRdmasPerConnection but the issue seems to
>     persist. Has anyone has encountered this and if so how'd you fix it?
>
>     -Aaron
>
>     --
>     Aaron Knister
>     NASA Center for Climate Simulation (Code 606.2)
>     Goddard Space Flight Center
>     (301) 286-2776 <tel:(301)%20286-2776>
>     _______________________________________________
>     gpfsug-discuss mailing list
>     gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
>     http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

--
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776


------------------------------

Message: 3
Date: Fri, 24 Feb 2017 19:39:30 +0000
From: Sven Oehme <oehmes at gmail.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] waiting for conn rdmas < conn maxrdmas
Message-ID:
        <CALssuR1nB3iL8eTBnqSsvnk=uOqWJqNgAC6w+10T6k+CpcNnmA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

its more likely you run out of verbsRdmasPerNode which is the top limit
across all connections for a given node.

Sven


On Fri, Feb 24, 2017 at 11:31 AM Aaron Knister <aaron.s.knister at nasa.gov>
wrote:

Interesting, thanks Sven!

Could "resources" I'm running out of include NSD server queues?

On 2/23/17 12:12 PM, Sven Oehme wrote:
> all this waiter shows is that you have more in flight than the node or
> connection can currently serve. the reasons for that can be
> misconfiguration or you simply run out of resources on the node, not the
> connection. with latest code you shouldn't see this anymore for node
> limits as the system automatically adjusts the number of maximum RDMA's
> according to the systems Node capabilities :
>
> you should see messages in your mmfslog like :
>
> 2017-02-23_06:19:50.056-0800: [I] VERBS RDMA starting with
> verbsRdmaCm=no verbsRdmaSend=yes verbsRdmaUseMultiCqThreads=yes
> verbsRdmaUseCompVectors=yes
> 2017-02-23_06:19:50.078-0800: [I] VERBS RDMA library libibverbs.so
> (version >= 1.1) loaded and initialized.
> 2017-02-23_06:19:50.078-0800: [I] VERBS RDMA verbsRdmasPerNode increased
> from*_3072 to 3740 because verbsRdmasPerNodeOptimize is set to yes._*
> 2017-02-23_06:19:50.121-0800: [I] VERBS RDMA discover mlx5_5 port 1
> transport IB link  IB NUMA node 16 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB9CD state ACTIVE
> 2017-02-23_06:19:50.137-0800: [I] VERBS RDMA discover mlx5_4 port 1
> transport IB link  IB NUMA node 16 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB9CC state ACTIVE
> 2017-02-23_06:19:50.153-0800: [I] VERBS RDMA discover mlx5_3 port 1
> transport IB link  IB NUMA node  1 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB751 state ACTIVE
> 2017-02-23_06:19:50.169-0800: [I] VERBS RDMA discover mlx5_2 port 1
> transport IB link  IB NUMA node  1 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB750 state ACTIVE
> 2017-02-23_06:19:50.185-0800: [I] VERBS RDMA discover mlx5_1 port 1
> transport IB link  IB NUMA node  0 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000013 id 0xE41D2D0300FDB78D state ACTIVE
> 2017-02-23_06:19:50.201-0800: [I] VERBS RDMA discover mlx5_0 port 1
> transport IB link  IB NUMA node  0 pkey[0] 0xFFFF gid[0] subnet
> 0xFEC0000000000015 id 0xE41D2D0300FDB78C state ACTIVE
>
> we want to eliminate all this configurable limits eventually, but this
> takes time, but as you can see above, we make progress on each release
:-)
>
> Sven
>
>
>
>
> On Thu, Feb 23, 2017 at 9:05 AM Aaron Knister <aaron.s.knister at nasa.gov
> <mailto:aaron.s.knister at nasa.gov>> wrote:
>
>     On a particularly heavy loaded NSD server I'm seeing a lot of these
>     messages:
>
>     0x7FFFF08B63E0 (  15539) waiting 0.004139456 seconds, NSDThread: on
>     ThCond 0x7FFFA80772C8 (0x7FFFA80772C8) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF08EED80 (  15584) waiting 0.004075718 seconds, NSDThread: on
>     ThCond 0x7FFF680008F8 (0x7FFF680008F8) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF08FDF00 (  15596) waiting 0.003965504 seconds, NSDThread: on
>     ThCond 0x7FFF8C00E288 (0x7FFF8C00E288) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF09185A0 (  15617) waiting 0.003916346 seconds, NSDThread: on
>     ThCond 0x7FFF9000CB18 (0x7FFF9000CB18) (VERBSEventWaitCondvar), reason
>     'waiting for conn rdmas < conn maxrdmas'
>     0x7FFFF092B380 (  15632) waiting 0.003659610 seconds, NSDThread: on
>     ThCond 0x1DB04B8 (0x1DB04B8) (VERBSEventWaitCondvar), reason 'waiting
>     for conn rdmas < conn maxrdmas'
>
>     I've tried tweaking verbsRdmasPerConnection but the issue seems to
>     persist. Has anyone has encountered this and if so how'd you fix it?
>
>     -Aaron
>
>     --
>     Aaron Knister
>     NASA Center for Climate Simulation (Code 606.2)
>     Goddard Space Flight Center
>     (301) 286-2776 <tel:(301)%20286-2776>
>     _______________________________________________
>     gpfsug-discuss mailing list
>     gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
>     http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

--
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20170224/c6db6493/attachment.html>

------------------------------

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


End of gpfsug-discuss Digest, Vol 61, Issue 46
**********************************************

________________________________

UT Southwestern


Medical Center



The future of medicine, today.



More information about the gpfsug-discuss mailing list