<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Just thinking: an application should do direct IO for a good
reason, and only then. "Forcing DIO" is probably not the right
thing to do - rather check why an app does DIO and either change
the app's behaviour if reasonable are maybe use a special pool
for it using mirrored SSDs or so. <br>
</p>
<p>BTW, the ESS have some nice mechanism to do small IOs (also
direct ones I suppose) quickly by buffering them on flash/NVRAM
(where the data is considered persistently stored, hence the IO
requests are completed quickly).</p>
<p>Uwe<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 12.03.24 11:59, Peter Hruška wrote:<br>
</div>
<blockquote type="cite"
cite="mid:7b539383643e14c1be8aafd9775b322eef0c22bc.camel@mcomputers.cz">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style>pre,code,address {
margin: 0px;
}h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}blockquote {
margin-top: 0em;
margin-bottom: 0em;
}</style>
<div>Hello,</div>
<div><br>
</div>
<div>The direct writes are problematic on both writes and
rewrites. Rewrites alone are another issue we have noticed.</div>
<div>Since indirect (direct=0) workloads are fine, it seems that
the easiest solution could be to force indirect IO operations
for all workloads. However we didn't find such possibility.<br>
</div>
<div><br>
</div>
<div><span>
<pre>--
</pre>
<div
style="font-size: 9pt; color: rgb(89, 89, 89); font-family: verdana;">S
přáním pěkného dne / Best regards</div>
<br
style="caret-color: rgb(46, 52, 54); color: rgb(46, 52, 54);">
<div
style="font-size: 11pt; color: rgb(220, 38, 30); font-family: verdana;"><b>Mgr.
Peter Hruška</b><br>
</div>
<div style="font-size: 9pt; font-family: verdana;"><font
color="#595959">IT specialista</font><br>
<br>
<div style="font-size: 9pt;"><span
style="color: rgb(89, 89, 89); font-size: 9pt; font-family: Verdana, sans-serif;"><b>M
Computers s.r.o.</b><br>
</span><span
style="color: rgb(220, 38, 30); font-size: 9pt; font-family: Verdana, sans-serif;">Úlehlova
3100/10, 628 00 Brno-Líšeň (<a
href="https://mapy.cz/s/gafufehufe"
title="Click to open https://mapy.cz/s/gafufehufe"
style="color: rgb(149, 79, 114);"
moz-do-not-send="true"><span
style="color: rgb(5, 99, 193);">mapa</span></a>)</span><span
style="font-size: 9pt; font-family: Verdana, sans-serif;"><br>
<font color="#595959">T:+420 515 538 136</font><br>
<font color="#595959">E: </font></span><a
href="mailto:peter.hruska@mcomputers.cz"
title="Click to mail peter.hruska@mcomputers.cz"
style="font-size: 9pt; color: rgb(149, 79, 114); font-family: Verdana, sans-serif;"
moz-do-not-send="true"><span
style="color: rgb(5, 99, 193);">peter.hruska@mcomputers.cz</span></a><span
style="font-size: 9pt; font-family: Verdana, sans-serif;"><font
color="#595959"><br>
</font><font color="#0563c1"><span
style="caret-color: rgb(5, 99, 193);"></span></font></span><span
style="font-size: 9pt; font-family: Verdana, sans-serif;"><br>
<a href="http://www.mcomputers.cz/"
title="Click to open http://www.mcomputers.cz/"
style="color: rgb(149, 79, 114);"
moz-do-not-send="true"><span
style="color: rgb(5, 99, 193);">www.mcomputers.cz</span></a><br>
</span></div>
</div>
<div><a href="http://www.lenovoshop.cz/"
title="Click to open http://www.lenovoshop.cz/"
style="font-family: Verdana, sans-serif; font-size: 9pt;"
moz-do-not-send="true">www.lenovoshop.cz</a></div>
<div><img src="cid:part1.x2mob0F7.ahCZvoxf@kit.edu" class=""
width="457px" height="106px"></div>
<div
style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(0, 0, 0, 0.4); -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 9pt; color: rgb(89, 89, 89); font-family: verdana;">
<br>
</div>
<div
style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Cantarell; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(0, 0, 0, 0.4); -webkit-text-stroke-width: 0px; text-decoration: none;">
<br>
</div>
</span></div>
<div><br>
</div>
<div>On Tue, 2024-03-12 at 09:59 +0100, Zdenek Salvet wrote:</div>
<blockquote type="cite"
style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>EXTERNÍ ODESÍLATEL<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On Mon, Mar 11, 2024 at 01:21:32PM +0000, Peter Hruška
wrote:<br>
</div>
<blockquote type="cite"
style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>We encountered a problem with performance of writes on
GPFS when the application uses direct io access. To simulate
the issue it is enough to run fio with option direct=1. The
performance drop is quite dramatic - 250 MiB/s vs. 2955
MiB/s. We've tried to instruct GPFS to ignore direct IO by
using "disableDIO=yes". The directive didn't have any
effect. Is there any possibility how to achieve that GPFS
would ignore direct IO requests and use caching for
everything?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Hello,<br>
</div>
<div>did you use pre-allocated file(s) (was it re-write) ?<br>
</div>
<div>libaio traffic is not really asynchronous with respect to
necessary metadata<br>
</div>
<div>operations (allocating new space and writing allocation
structures to disk)<br>
</div>
<div>in most Linux filesystems and I guess this case is not
heavily optimized<br>
</div>
<div>in GPFS either (dioSmallSeqWriteBatching feature may help a
little but<br>
</div>
<div>it targets different scenario I think).<br>
</div>
<div><br>
</div>
<div>Best regards,<br>
</div>
<div>Zdenek Salvet
<a href="mailto:salvet@ics.muni.cz" moz-do-not-send="true"
class="moz-txt-link-freetext">
salvet@ics.muni.cz</a><br>
</div>
<div>Institute of Computer Science of Masaryk University, Brno,
Czech Republic<br>
</div>
<div>and CESNET, z.s.p.o., Prague, Czech Republic<br>
</div>
<div>Phone: ++420-549 49 6534 Fax:
++420-541 212 747<br>
</div>
<div>----------------------------------------------------------------------------<br>
</div>
<div> Teamwork is essential -- it allows you to blame
someone else.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>_______________________________________________<br>
</div>
<div>gpfsug-discuss mailing list<br>
</div>
<div>gpfsug-discuss at gpfsug.org<br>
</div>
<div><a
href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org"
moz-do-not-send="true" class="moz-txt-link-freetext">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a><br>
</div>
</blockquote>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Karlsruhe Institute of Technology (KIT)
Scientific Computing Centre (SCC)
Scientific Data Management (SDM)
Uwe Falke
Hermann-von-Helmholtz-Platz 1, Building 442, Room 187
D-76344 Eggenstein-Leopoldshafen
Tel: +49 721 608 28024
Email: <a class="moz-txt-link-abbreviated" href="mailto:uwe.falke@kit.edu">uwe.falke@kit.edu</a>
<a class="moz-txt-link-abbreviated" href="http://www.scc.kit.edu">www.scc.kit.edu</a>
Registered office:
Kaiserstraße 12, 76131 Karlsruhe, Germany
KIT – The Research University in the Helmholtz Association
</pre>
</body>
</html>