[gpfsug-discuss] Slow performance on writes when using direct io

[Zdenek Salvet]

On Mon, Mar 11, 2024 at 01:21:32PM +0000, Peter Hruška wrote:
> We encountered a problem with performance of writes on GPFS when the application uses direct io access. To simulate the issue it is enough to run fio with option direct=1. The performance drop is quite dramatic - 250 MiB/s vs. 2955 MiB/s. We've tried to instruct GPFS to ignore direct IO by using "disableDIO=yes". The directive didn't have any effect. Is there any possibility how to achieve that GPFS would ignore direct IO requests and use caching for everything?

Hello,
did you use pre-allocated file(s) (was it re-write) ? 
libaio traffic is not really asynchronous with respect to necessary metadata
operations (allocating new space and writing allocation structures to disk)
in most Linux filesystems and I guess this case is not heavily optimized
in GPFS either (dioSmallSeqWriteBatching feature may help a little but
it targets different scenario I think).

Best regards,
Zdenek Salvet                                              salvet at ics.muni.cz 
Institute of Computer Science of Masaryk University, Brno, Czech Republic
and CESNET, z.s.p.o., Prague, Czech Republic
Phone: ++420-549 49 6534                           Fax: ++420-541 212 747
----------------------------------------------------------------------------
      Teamwork is essential -- it allows you to blame someone else.