[gpfsug-discuss] QoS question
Marc A Kaplan
makaplan at us.ibm.com
Wed Jun 15 19:24:07 BST 2016
Thanks for that reference. Further research indicates that GRIO on XFS
required Irix/SGI hardware features and GRIO is not in any recent
implementations of XFS.
Functionally GRIO, when it worked was kind of a logical complement or
opposite of GPFS/QOS. GRIO was to be used to reserve bandwidth for
critical applications; whereas GPFS 4.2/QOS primary purpose to restrict
the IO bandwidth consumed by not-performance critical applications. Of
course if you restrict set X to use no more than B, than you have
effectively reserved TotalAvailable-B for the set ~X. It may interest you
to know that the primary GPFS/QOS enforcement mechanisms are based on an
implementation of Token Bucket ( wikipedia.org/wiki/Token_bucket , US
Patent 9178827 )
--marc
From: Jan-Frode Myklebust <janfrode at tanso.net>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 06/15/2016 01:55 AM
Subject: Re: [gpfsug-discuss] QoS question
Sent by: gpfsug-discuss-bounces at spectrumscale.org
XFS on Irix had a feature similar to QoS, called GRIO (guaranteed rate
I/O), where applications could reserve a given bandwidth.
http://www.sgistuff.net/software/irixintro/documents/xfs-whitepaper.html
Sounds somewhat similar to QoS, but focused on giving applications
guaranteed bandwidth, not iops.
-jf
ons. 15. jun. 2016 kl. 00.08 skrev Marc A Kaplan <makaplan at us.ibm.com>:
Yes, in QOS for 4.2.0 there are some simple assumptions that may not make
a lot of sense in some configurations, especially configurations with many
(100s) of nodes mounting the same file system... You can try out what
you suggested and in 4.2.0 I think it will pretty much work as you suggest
-- essentially you are allocating 466 maintenance iops to every node,
knowing that most of those nodes will not be using their allocation of
IOPS.
In later releases, you may find that we will address some of these kinds
of quirks in QOS.
QOS is a new feature for GPFS, and I don't think you'll find anything like
it in any commercial file system offering. (Correct me with example(s) if
I am wrong on this point.)
So think of it as "release 1.0" (of QOS) and let us know how well it works
for you and how it might be improved....
--marc of Spectrum(GP)FS
From: "Buterbaugh, Kevin L" <Kevin.Buterbaugh at Vanderbilt.Edu>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 06/14/2016 04:50 PM
Subject: [gpfsug-discuss] QoS question
Sent by: gpfsug-discuss-bounces at spectrumscale.org
Hi All,
We have recently upgraded to GPFS 4.2.0-3 and so I am getting ready to
dive into my first attempts at using the new QoS features. I want to make
sure I am understanding the documentation:
"The IOPS values that you set in an mmchqos command apply to all I/O
operations that are issued by all the nodes that have the specified file
system mounted. You should adjust your allocations of IOPS accordingly.
For example, if you 600 IOPS to the maintenance class, and there are six
nodes that have the file system mounted, then QoS allocates 100 IOPS to
the maintenance class of each node. If you then run maintenance commands
that affect only three of the nodes, the commands runs with an actual
allocation of 300 IOPS, or 100 IOPS per node. To run maintenance commands
that affect three nodes with an actual allotment of 600 IOPS, or 200 per
node, allocate 1200 IOPS to the maintenanceclass. "
We have a ~700 node cluster with 15 NSD servers. Here’s how I interpret
the above assuming that I have determined that I want to allow 7,000 IOPs
… please correct me if I’m wrong...
7,000 IOPs / 700 nodes would be 10 IOPs per node.
But I want those 7,000 IOPs to be divided amongst my 15 NSD servers that
are going to be doing the maintenance (a restripe, for example), so 7,000
/ 15 = 466.67.
466.67 * 700 (nodes in cluster) = 326,666.67. So I would allocate 326,666
IOPs to the maintenance class?
Thanks in advance…
Kevin
—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and
Education
Kevin.Buterbaugh at vanderbilt.edu- (615)875-9633
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160615/10088bc2/attachment-0002.htm>
More information about the gpfsug-discuss
mailing list