[gpfsug-discuss] mmcrfs issue

[Aaron Knister]

On 3/14/17 2:15 PM, valdis.kletnieks at vt.edu wrote:
> On Mon, 13 Mar 2017 20:06:29 -0400, Aaron Knister said:
>> After setting the sync=always parameter to not lose data in the event of
>> a crash or power outage the write performance became unbearably slow
>> (under 100MB/s of writes for an 8+2 RAIDZ2 if I recall correctly). I
>
> Not at all surprising. Forcing a 'sync every write' has been painful for pretty
> much everything - first time I saw it was on a SunOS 3.2 box doing NFS over 3
> decades ago.
>

You know, what's interesting is when you do direct I/O to a linux md 
RAID device each acknowledged I/O to the md device results in SCSI 
SYNCHRONIZE CACHE commands being sent to the underlying member devices. 
I consider that doing a sync of sorts after each write and as long as 
everything is properly aligned you can get pretty fantastic performance 
from this. No data checksums, though :(

>> I've got an RFE open with IBM
>> (https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=84994)
>> to see if the behavior of GPFS could be changed such that it would issue
>> explicit cache flushes that would allow it to work with ZFS (it might
>> even be beneficial in FPO environments too).
>
> Do you have a suggestion for how to do this without it turning into 'sync=always'
> just done at a different layer in the stack?
>

I don't think that GPFS would need to issue a sync after *every* write. 
I think the key is more that GPFS is aware of the state of acknowledged 
but uncommitted writes so that if there's something *really* important 
to be written (i.e. an internal piece of metadata) GPFS can force it to 
be committed to avoid GPFS itself becoming inconsistent. You don't want 
to have to mmfsck after a power outage/crash if there were lost writes 
that GPFS assumed were committed.

-Aaron

>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: OpenPGP digital signature
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170314/73c5eb59/attachment-0002.sig>