<div dir="ltr">Hi Ken,<div><br></div><div>what the documents is saying (or try to) is that the behavior of data in inode or metadata operations are not changed if HAWC is enabled, means if the data fits into the inode it will be placed there directly instead of writing the data i/o into a data recovery log record (which is what HAWC uses) and then later destage it where ever the data blocks of a given file eventually will be written. that also means if all your application does is creating small files that fit into the inode, HAWC will not be able to improve performance.</div><div>its unfortunate not so simple to say if HAWC will help or not, but here are a couple of thoughts where HAWC will not help and help : </div><div>on the where it won't help : </div><div>1. if you have storage device which has very large or even better are log structured write cache.</div><div>2. if majority of your files are very small</div><div>3. if your files will almost always be accesses sequentially </div><div>4. your storage is primarily flash based</div><div>where it most likely will help : </div><div>1. your majority of storage is direct attached HDD (e.g. FPO) with a small SSD pool for metadata and HAWC</div><div>2. your ratio of clients to storage devices is very high (think hundreds of clients and only 1 storage array) </div><div>3. your workload is primarily virtual machines or databases </div><div><br></div><div>as always there are lots of exceptions and corner cases, but is the best list i could come up with. </div><div><br></div><div>on how to find out if HAWC could help, there are 2 ways of doing this </div><div>first, look at mmfsadm dump iocounters , you see the average size of i/os and you could check if there is a lot of small write operations done. </div><div>a more involved but more accurate way would be to take a trace with trace level trace=io , that will generate a very lightweight trace of only the most relevant io layers of GPFS, you could then post process the operations performance, but the data is not the simplest to understand for somebody with low knowledge of filesystems, but if you stare at it for a while it might make some sense to you. </div><div><br></div><div>Sven<br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Sep 3, 2018 at 4:06 PM Kenneth Waegeman <<a href="mailto:kenneth.waegeman@ugent.be">kenneth.waegeman@ugent.be</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Thank you Vasily and Simon for the clarification!</p>
<p>I was looking further into it, and I got stuck with more
questions :)</p>
<p><br>
- In
<a class="m_2333020818233895813moz-txt-link-freetext" href="https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_hawc_tuning.htm" target="_blank">https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_hawc_tuning.htm</a>
I read:<br>
HAWC does not change the following behaviors:<br>
write behavior of small files when the data is placed in
the inode itself<br>
write behavior of directory blocks or other metadata<br>
</p>
I wondered why? Is the metadata not logged in the (same) recovery
logs? (It seemed by reading
<a class="m_2333020818233895813moz-txt-link-freetext" href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.ins.doc/bl1ins_logfile.htm" target="_blank">https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.ins.doc/bl1ins_logfile.htm</a>
it does )<br>
<br>
<br>
- Would there be a way to estimate how much of the write requests on
a running cluster would benefit from enabling HAWC ?<br>
<br>
<br>
Thanks again!</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
Kenneth</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<div class="m_2333020818233895813moz-cite-prefix">On 31/08/18 19:49, Vasily Tarasov
wrote:<br>
</div>
<blockquote type="cite">
<div class="m_2333020818233895813socmaildefaultfont" dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10.5pt">
<div dir="ltr">That is correct. The blocks of each recovery log
are striped across the devices in the system.log pool (if it
is defined). As a result, even when all clients have a local
device in the system.log pool, many writes to the recovery log
will go to remote devices. For a client that lacks a local
device in the system.log pool, log writes will always be
remote.</div>
<div dir="ltr"> </div>
<div dir="ltr">Notice, that typically in such a setup you would
enable log replication for HA. Otherwise, if a single client
fails (and its recover log is lost) the whole cluster fails as
there is no log to recover FS to consistent state. Therefore,
at least one remote write is essential.</div>
<div dir="ltr"> </div>
<div dir="ltr">HTH,</div>
<div dir="ltr">
<div class="m_2333020818233895813socmaildefaultfont" dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10.5pt">
<div class="m_2333020818233895813socmaildefaultfont" dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10.5pt">
<div dir="ltr">
<div>--</div>
<div>Vasily Tarasov,</div>
<div>Research Staff Member,</div>
<div>Storage Systems Research,</div>
<div>IBM Research - Almaden</div>
</div>
</div>
</div>
</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
<blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px">-----
Original message -----<br>
From: Kenneth Waegeman <a class="m_2333020818233895813moz-txt-link-rfc2396E" href="mailto:kenneth.waegeman@ugent.be" target="_blank"><kenneth.waegeman@ugent.be></a><br>
Sent by: <a class="m_2333020818233895813moz-txt-link-abbreviated" href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a><br>
To: gpfsug main discussion list
<a class="m_2333020818233895813moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank"><gpfsug-discuss@spectrumscale.org></a><br>
Cc:<br>
Subject: [gpfsug-discuss] system.log pool on client nodes for
HAWC<br>
Date: Tue, Aug 28, 2018 5:31 AM<br>
<div><font size="2" face="Default Monospace,Courier
New,Courier,monospace">Hi all,<br>
<br>
I was looking into HAWC , using the 'distributed fast
storage in client<br>
nodes' method (<br>
<a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_hawc_using.htm" target="_blank">https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_hawc_using.htm</a> <br>
)<br>
<br>
This is achieved by putting a local device on the clients
in the<br>
system.log pool. Reading another article<br>
(<a href="https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_syslogpool.htm" target="_blank">https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_syslogpool.htm</a> <br>
) this would now be used for ALL File system recovery
logs.<br>
<br>
Does this mean that if you have a (small) subset of
clients with fast<br>
local devices added in the system.log pool, all other
clients will use<br>
these too instead of the central system pool?<br>
<br>
Thank you!<br>
<br>
Kenneth<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font><br>
</div>
</blockquote>
<div dir="ltr"> </div>
</div>
<br>
<br>
<fieldset class="m_2333020818233895813mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a>
<a class="m_2333020818233895813moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div>