[gpfsug-discuss] mmfsd write behavior
Sven Oehme
oehmes at gmail.com
Fri Sep 8 22:21:00 BST 2017
Hi,
the code assumption is that the underlying device has no volatile write
cache, i was absolute sure we have that somewhere in the FAQ, but i
couldn't find it, so i will talk to somebody to correct this.
if i understand
https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt
correct
one could enforce this by setting REQ_FUA, but thats not explicitly set
today, at least i can't see it. i will discuss this with one of our devs
who owns this code and come back.
sven
On Thu, Sep 7, 2017 at 8:05 PM Aaron Knister <aaron.s.knister at nasa.gov>
wrote:
> Thanks Sven. I didn't think GPFS itself was caching anything on that
> layer, but it's my understanding that O_DIRECT isn't sufficient to force
> I/O to be flushed (e.g. the device itself might have a volatile caching
> layer). Take someone using ZFS zvol's as NSDs. I can write() all day log
> to that zvol (even with O_DIRECT) but there is absolutely no guarantee
> those writes have been committed to stable storage and aren't just
> sitting in RAM until an fsync() occurs (or some other bio function that
> causes a flush). I also don't believe writing to a SATA drive with
> O_DIRECT will force cache flushes of the drive's writeback cache..
> although I just tested that one and it seems to actually trigger a scsi
> cache sync. Interesting.
>
> -Aaron
>
> On 9/7/17 10:55 PM, Sven Oehme wrote:
> > I am not sure what exactly you are looking for but all blockdevices are
> > opened with O_DIRECT , we never cache anything on this layer .
> >
> >
> > On Thu, Sep 7, 2017, 7:11 PM Aaron Knister <aaron.s.knister at nasa.gov
> > <mailto:aaron.s.knister at nasa.gov>> wrote:
> >
> > Hi Everyone,
> >
> > This is something that's come up in the past and has recently
> resurfaced
> > with a project I've been working on, and that is-- it seems to me as
> > though mmfsd never attempts to flush the cache of the block devices
> its
> > writing to (looking at blktrace output seems to confirm this). Is
> this
> > actually the case? I've looked at the gpl headers for linux and I
> don't
> > see any sign of blkdev_fsync, blkdev_issue_flush, WRITE_FLUSH, or
> > REQ_FLUSH. I'm sure there's other ways to trigger this behavior that
> > GPFS may very well be using that I've missed. That's why I'm asking
> :)
> >
> > I figure with FPO being pushed as an HDFS replacement using commodity
> > drives this feature has *got* to be in the code somewhere.
> >
> > -Aaron
> >
> > --
> > Aaron Knister
> > NASA Center for Climate Simulation (Code 606.2)
> > Goddard Space Flight Center
> > (301) 286-2776
> > _______________________________________________
> > gpfsug-discuss mailing list
> > gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
> > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> >
> >
> >
> > _______________________________________________
> > gpfsug-discuss mailing list
> > gpfsug-discuss at spectrumscale.org
> > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> >
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170908/b2985540/attachment-0002.htm>
More information about the gpfsug-discuss
mailing list