[gpfsug-discuss] gpfsug-discuss Digest, Vol 116, Issue 6

Jon Diprose jon at well.ox.ac.uk
Mon Sep 20 14:40:17 BST 2021

Thanks Madhav. I am aware of that, and find myself with a need for krb5p - and a disappointed user. I am hoping for some rough quantification of the expected impact of turning on end-to-ed encryption so I know whether what I get is all there is or whether I need to keep digging.
I know that Spectrum Scale uses the AES-NI instructions (if available) for its own end-to-end encryption. I am less clear on whether AES-NI is used by ganesha for krb5p. Both ends have cpus that indicate support of AES-NI to the OS. I can, and with apologies for it being a pdf, point you at a paper on AES-NI performance:


which shows that their Intel i5-8250U test platform (which is probably disappointingly close in single-thread performance to my 6212U-based servers) will happily push around 900MB/s using AES-NI accelaration, but only around 110MB/s without. The higher number wouldn't be the bottleneck in my setup. The lower number is pretty close to what I am seeing. Unfortunately I can't tell whether AES-NI is actually being used, or find any options that might control its use.
Do you have any numbers to indicate what throughput I might expect to get for krb5p, and what hit that might be over krb5? Or any suggestions for checking whether AES-NI is actually in use?

Dr. Jonathan Diprose <jon at well.ox.ac.uk<mailto:jon at well.ox.ac.uk>>             Tel: 01865 287873
Research Computing Manager
Henry Wellcome Building for Genomic Medicine
Roosevelt Drive, Headington, Oxford OX3 7BN
From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of Madhav Ponamgi1 [mzp at us.ibm.com]
Sent: 20 September 2021 13:44
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 116, Issue 6

There are 3 flavors of NFS Kerberos (I'm only going to address NFS 4.x):
Krb5 - encrypts authentication
Krtbi - encrypts authentication and provides checksums (reducing man-in-the-middle attacks)
Krb5p - End-to-end encryption with integrity checking

The Krb5p protocol provides ultimate security but comes at a cost where all NFS packets will be encrypted (mount authenticated) and with checksums.   This
can add considerable overhead (for example, using AES-256 is similar to SMB3 signing and sealing).   There are AES-NI off-loading engines to reduce this
overhead.   So it is not surprising to see significant performance drop when using Krb5p versus Krb5.

Madhav Ponamgi
mzp at us.ibm.com
(215) 794-6987
Tech Sales Website:  w3.ibm.com/w3publisher/ww_storage_tech_sales

From:        gpfsug-discuss-request at spectrumscale.org
To:        gpfsug-discuss at spectrumscale.org
Date:        09/20/2021 07:00 AM
Subject:        [EXTERNAL] gpfsug-discuss Digest, Vol 116, Issue 6
Sent by:        gpfsug-discuss-bounces at spectrumscale.org

Send gpfsug-discuss mailing list submissions to
                gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
or, via email, send a message with subject or body 'help' to
                gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
                gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."

Today's Topics:

  1. nfs krb5p performance (Jon Diprose)


Message: 1
Date: Mon, 20 Sep 2021 09:58:02 +0000
From: Jon Diprose <jon at well.ox.ac.uk>
To: "gpfsug-discuss at spectrumscale.org"
                <gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] nfs krb5p performance
                <CF41F7F23121954A8E819732615C61257AAE3DDB at exchange01.well.ox.ac.uk>
Content-Type: text/plain; charset="us-ascii"

We have just started using the nfs protocol with SECTYPE=krb5p and are a little surprised by the performance impact - looks like down to a third of that of SECTYPE=krb5. Would any of you using krb5p be kind enough to share your estimates of impact? Not sure if we have a misconfiguration of setup or expectation.

Dr. Jonathan Diprose <jon at well.ox.ac.uk>             Tel: 01865 287873
Research Computing Manager
Henry Wellcome Building for Genomic Medicine
Roosevelt Drive, Headington, Oxford OX3 7BN


gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org

End of gpfsug-discuss Digest, Vol 116, Issue 6

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210920/61a84c94/attachment-0002.htm>

More information about the gpfsug-discuss mailing list