[gpfsug-discuss] Spectrum Scale 5 and Reading Compressed Data

Jan-Frode Myklebust janfrode at tanso.net
Wed Jan 20 14:47:07 GMT 2021


This sounds like a bug to me... (I wouldn't expect mmchattr works on
different node than other file access). I would check "mmdiag --iohist
verbose" during these slow reads, to see if it gives a hint at what it's
doing, versus what it shows during "mmchattr". Maybe one is
triggering prefetch, while the other is some kind of random IO ? Also might
be worth to try a mmtrace. Compare the traces for

mmtrace start trace="all 0 vnode 1 vnop 1 io 1"
cat compressedLargeFile
mmtrace stop

vs.:

mmtrace start trace="all 0 vnode 1 vnop 1 io 1"
mmchattr --compress no someLargeFile
mmtrace stop

(but please make sure that the file wasn't already uncompressed in pagepool
in this second run).


  -jf

On Wed, Jan 20, 2021 at 12:59 PM Daniel Kidger <daniel.kidger at uk.ibm.com>
wrote:

> I think you need to think about which node the file is being decompressed
> on (and if that node has plenty of space in the page pool.)
> iirc mmchattr works on one of the 'manager' nodes not necessarily the node
> you typed the command on?
> Daniel
>
> _________________________________________________________
> *Daniel Kidger Ph.D.*
> IBM Technical Sales Specialist
> Spectrum Scale, Spectrum Discover  and IBM Cloud Object Storage
>
> +44-(0)7818 522 266
> daniel.kidger at uk.ibm.com
>
> <https://www.youracclaim.com/badges/687cf790-fe65-4a92-b129-d23ae41862ac/public_url>
> <https://www.youracclaim.com/badges/8dac4bc0-7b3a-4035-b127-daec8dce9200>
> <https://www.youracclaim.com/badges/8153c6a7-3e02-40be-87ee-24e27ae9459c/public_url>
> <https://www.youracclaim.com/badges/78197e2c-4277-4ec9-808b-ad6abe1e1b16/public_url>
>
>
>
>
> ----- Original message -----
> From: Alec <anacreo at gmail.com>
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> To: gpfsug-discuss at spectrumscale.org
> Cc:
> Subject: [EXTERNAL] [gpfsug-discuss] Spectrum Scale 5 and Reading
> Compressed Data
> Date: Wed, Jan 20, 2021 11:10
>
> We have AIX and Spectrum Scale 5.1 and are compressing older data.
>
> We can compress data at about 10GB/minute and decompress data wicked fast
> using mmchattr, when a user reads data from a compressed file via
> application open / read calls.... it moves at about 5MB/s.  Normally our
> I/O pipeline allows for 2400MB/s on a single file read.
>
> What can we look at to speed up the read of the compressed data, are there
> any tunables that might affect this?
>
> As it is now if the backup daemon is backing up a compressed file, it can
> get stuck for hours, I will go and mmchattr to decompress the file, within
> a minute the file is decompressed, and backed up, then I simply recompress
> the file once backup has moved on.
>
> Any advice on how to improve the compressed reads under AIX would be very
> helpful.
>
> Alec
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=USgQqOp8HDCg0DYjdjSVFvVOwq1rMgRYPP_hoZqgUyI&s=_hdEB3EvWW-8ZzdS1D1roh92-AicdrVMywJwQGlKTIQ&e=
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210120/8918644b/attachment-0002.htm>


More information about the gpfsug-discuss mailing list