<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I am not 100% sure what the workload of the VMs is. We have 100's of
VMs all used differently, so the workload is rather mixed. <br>
<br>
I do see 4K writes going to "system" pool, they are tagged as
"logData" in 'mmdiag --iohist'. But I also see 4K writes going to
the data drives, so it looks like everything is not getting
coalesced and these are random writes. <br>
<br>
Could these 4k writes labelled as "logData" be the writes going to
HAWC log files?<br>
<br>
<br>
<div class="moz-cite-prefix">On 8/1/2016 15:50, Dean Hildebrand
wrote:<br>
</div>
<blockquote
cite="mid:OFABC8B4ED.2CF1257D-ON88258002.006BA225-88258002.006CF637@notes.na.collabserv.com"
type="cite">
<p>Hi Tejas,<br>
<br>
Do you know the workload in the VM? <br>
<br>
The workload which enters into HAWC may or may not be the same
as the workload that eventually goes into the data pool....it
all depends on whether the 4KB writes entering HAWC can be
coalesced or not. For example, sequential 4KB writes can all be
coalesced into a single large chunk. So 4KB writes into HAWC
will convert into 8MB writes to data pool (in your system). But
random 4KB writes into HAWC may end up being 4KB writes into the
data pool if there are no adjoining 4KB writes (i.e., if 4KB
blocks are all dispersed, they can't be coalesced). The goal of
HAWC though, whether the 4KB blocks are coalesced or not, is to
reduce app latency by ensuring that writing the blocks back to
the data pool is done in the background. So while 4KB blocks may
still be hitting the data pool, hopefully the application is
seeing the latency of your presumably lower latency system pool.<br>
<br>
Dean <br>
<br>
<br>
<img src="cid:part1.1DA50A86.80E6ACFD@bnl.gov" alt="Inactive
hide details for Tejas Rao ---08/01/2016 12:06:15 PM---In my
case GPFS storage is used to store VM images (KVM) and he"
height="16" border="0" width="16"><font color="#424282">Tejas
Rao ---08/01/2016 12:06:15 PM---In my case GPFS storage is
used to store VM images (KVM) and hence the small IO.</font><br>
<br>
<font color="#5F5F5F" size="2">From: </font><font size="2">Tejas
Rao <a class="moz-txt-link-rfc2396E" href="mailto:raot@bnl.gov"><raot@bnl.gov></a></font><br>
<font color="#5F5F5F" size="2">To: </font><font size="2">gpfsug
main discussion list <a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss@spectrumscale.org"><gpfsug-discuss@spectrumscale.org></a></font><br>
<font color="#5F5F5F" size="2">Date: </font><font size="2">08/01/2016
12:06 PM</font><br>
<font color="#5F5F5F" size="2">Subject: </font><font size="2">Re:
[gpfsug-discuss] HAWC (Highly available write cache)</font><br>
<font color="#5F5F5F" size="2">Sent by: </font><font size="2"><a class="moz-txt-link-abbreviated" href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a></font><br>
</p>
<hr style="color:#8091A5; " align="left" noshade="noshade"
size="2" width="100%"><br>
<br>
<br>
<font size="4">In my case GPFS storage is used to store VM images
(KVM) and hence the small IO.<br>
<br>
I always see lots of small 4K writes and the GPFS filesystem
block size is 8MB. I thought the reason for the small writes is
that the linux kernel requests GPFS to initiate a periodic sync
which by default is every 5 seconds and can be controlled by
"vm.dirty_writeback_centisecs". <br>
<br>
I thought HAWC would help in such cases and would harden
(coalesce) the small writes in the "system" pool and would flush
to the "data" pool in larger block size. <br>
<br>
Note - I am not doing direct i/o explicitly. <br>
<br>
<br>
</font><br>
<font size="4">On 8/1/2016 14:49, Sven Oehme wrote:</font>
<ul>
<ul>
<font size="4">when you say 'synchronous write' what do you
mean by that ? </font><br>
<font size="4">if you are talking about using direct i/o
(O_DIRECT flag), they don't leverage HAWC data path, its by
design.</font><br>
<br>
<font size="4">sven</font><br>
<br>
<font size="4">On Mon, Aug 1, 2016 at 11:36 AM, Tejas Rao <</font><a
moz-do-not-send="true" href="mailto:raot@bnl.gov"
target="_blank"><u><font color="#0000FF" size="4">raot@bnl.gov</font></u></a><font
size="4">> wrote:</font>
<ul>
<font size="4">I have enabled write cache (HAWC) by running
the below commands. The recovery logs are supposedly
placed in the replicated system metadata pool (SSDs). I do
not have a "system.log" pool as it is only needed if
recovery logs are stored on the client nodes.<br>
<br>
mmchfs gpfs01 --write-cache-threshold 64K<br>
mmchfs gpfs01 -L 1024M<br>
mmchconfig logPingPongSector=no<br>
<br>
I have recycled the daemon on all nodes in the cluster
(including the NSD nodes).<br>
<br>
I still see small synchronous writes (4K) from the clients
going to the data drives (data pool). I am checking this
by looking at "mmdiag --iohist" output. Should they not be
going to the system pool?<br>
<br>
Do I need to do something else? How can I confirm that
HAWC is working as advertised?<br>
<br>
Thanks.<br>
<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at </font><a moz-do-not-send="true"
href="http://spectrumscale.org/" target="_blank"><u><font
color="#0000FF" size="4">spectrumscale.org</font></u></a><u><font
color="#0000FF" size="4"><br>
</font></u><a moz-do-not-send="true"
href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"
target="_blank"><u><font color="#0000FF" size="4">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></u></a>
</ul>
<br>
<font size="4"><br>
</font><br>
<tt><font size="4">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
</font></tt><a moz-do-not-send="true"
href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><u><font
color="#0000FF" size="4">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></u></tt></a><tt><font
size="4"><br>
</font></tt>
</ul>
</ul>
<tt>_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
</tt><tt><a moz-do-not-send="true"
href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br>
</tt><br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>