<div dir="ltr">
<p class="inbox-inbox-p1">end-to-end data integrity is very important and the reason it hasn't been done in Scale is not because its not important, its because its very hard to do without impacting performance in a very dramatic way.<span class="inbox-inbox-Apple-converted-space"> </span></p>
<p class="inbox-inbox-p1">imagine your raid controller blocksize is 1mb and your filesystem blocksize is 1MB . if your application does a 1 MB write this ends up being a perfect full block , full track de-stage to your raid layer and everything works fine and fast. as soon as you add checksum support you need to add data somehow into this, means your 1MB is no longer 1 MB but 1 MB+checksum.<span class="inbox-inbox-Apple-converted-space"> </span></p>
<p class="inbox-inbox-p1">to store this additional data you have multiple options, inline , outside the data block or some combination ,the net is either you need to do more physical i/o's to different places to get both the data and the corresponding checksum or your per block on disc structure becomes bigger than than what your application reads/or writes, both put massive burden on the Storage layer as e.g. a 1 MB write will now, even the blocks are all aligned from the application down to the raid layer, cause a read/modify/write on the raid layer as the data is bigger than the physical track size.<span class="inbox-inbox-Apple-converted-space"> </span></p>
<p class="inbox-inbox-p2">so to get end-to-end checksum in Scale outside of ESS the best way is to get GNR as SW to run on generic HW, this is what people should vote for as RFE if they need that functionality. beside end-to-end checksums you get read/write cache and acceleration , fast rebuild and many other goodies as a added bonus. <br></p>
<p class="inbox-inbox-p2">Sven<br></p></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Jun 30, 2017 at 10:53 AM Aaron Knister <<a href="mailto:aaron.s.knister@nasa.gov">aaron.s.knister@nasa.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In fact the answer was quite literally "no":<br>
<br>
<a href="https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=84523" rel="noreferrer" target="_blank">https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=84523</a><br>
(the RFE was declined and the answer was that the "function is already<br>
available in GNR environments").<br>
<br>
Regarding GNR, see this RFE request<br>
<a href="https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=95090" rel="noreferrer" target="_blank">https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=95090</a><br>
requesting the use of GNR outside of an ESS/GSS environment. It's<br>
interesting to note this is the highest voted Public RFE for GPFS that I<br>
can see, at least. It too was declined.<br>
<br>
-Aaron<br>
<br>
On 6/30/17 1:41 PM, Aaron Knister wrote:<br>
> Thanks Olaf, that's good to know (and is kind of what I suspected). I've<br>
> requested a number of times this capability for those of us who can't<br>
> use or aren't using GNR and the answer is effectively "no". This<br>
> response is curious to me because I'm sure IBM doesn't believe that data<br>
> integrity is only important and of value to customers who purchase their<br>
> hardware *and* software.<br>
><br>
> -Aaron<br>
><br>
> On Fri, Jun 30, 2017 at 1:37 PM, Olaf Weiser <<a href="mailto:olaf.weiser@de.ibm.com" target="_blank">olaf.weiser@de.ibm.com</a><br>
> <mailto:<a href="mailto:olaf.weiser@de.ibm.com" target="_blank">olaf.weiser@de.ibm.com</a>>> wrote:<br>
><br>
> yes.. in case of GNR (GPFS native raid) .. we do end-to-end<br>
> check-summing ... client --> server --> downToDisk<br>
> GNR writes down a chksum to disk (to all pdisks /all "raid" segments<br>
> ) so that dropped writes can be detected as well as miss-done<br>
> writes (bit flips..)<br>
><br>
><br>
><br>
> From: Aaron Knister <<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a><br>
> <mailto:<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a>>><br>
> To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>
> <mailto:<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>>><br>
> Date: 06/30/2017 07:15 PM<br>
> Subject: [gpfsug-discuss] Fwd: FLASH: IBM Spectrum Scale (GPFS):<br>
> RDMA-enabled network adapter failure on the NSD server may result in<br>
> file IO error (2017.06.30)<br>
> Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a><br>
> <mailto:<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>><br>
> ------------------------------------------------------------------------<br>
><br>
><br>
><br>
> I'm curious to know why this doesn't affect GSS/ESS? Is it a feature of<br>
> the additional check-summing done on those platforms?<br>
><br>
><br>
> -------- Forwarded Message --------<br>
> Subject: FLASH: IBM Spectrum Scale (GPFS): RDMA-enabled network<br>
> adapter<br>
> failure on the NSD server may result in file IO error (2017.06.30)<br>
> Date: Fri, 30 Jun 2017 14:19:02 +0000<br>
> From: IBM My Notifications<br>
> <<a href="mailto:mynotify@stg.events.ihost.com" target="_blank">mynotify@stg.events.ihost.com</a> <mailto:<a href="mailto:mynotify@stg.events.ihost.com" target="_blank">mynotify@stg.events.ihost.com</a>>><br>
> To: <a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a> <mailto:<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a>><br>
><br>
><br>
><br>
><br>
> My Notifications for Storage - 30 Jun 2017<br>
><br>
> Dear Subscriber (<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a><br>
> <mailto:<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a>>),<br>
><br>
> Here are your updates from IBM My Notifications.<br>
><br>
> Your support Notifications display in English by default. Machine<br>
> translation based on your IBM profile<br>
> language setting is added if you specify this option in My defaults<br>
> within My Notifications.<br>
> (Note: Not all languages are available at this time, and the English<br>
> version always takes precedence<br>
> over the machine translated version.)<br>
><br>
> ------------------------------------------------------------------------------<br>
> 1. IBM Spectrum Scale<br>
><br>
> - TITLE: IBM Spectrum Scale (GPFS): RDMA-enabled network adapter<br>
> failure<br>
> on the NSD server may result in file IO error<br>
> - URL:<br>
> <a href="http://www.ibm.com/support/docview.wss?uid=ssg1S1010233&myns=s033&mynp=OCSTXKQY&mynp=OCSWJ00&mync=E&cm_sp=s033-_-OCSTXKQY-OCSWJ00-_-E" rel="noreferrer" target="_blank">http://www.ibm.com/support/docview.wss?uid=ssg1S1010233&myns=s033&mynp=OCSTXKQY&mynp=OCSWJ00&mync=E&cm_sp=s033-_-OCSTXKQY-OCSWJ00-_-E</a><br>
> <<a href="http://www.ibm.com/support/docview.wss?uid=ssg1S1010233&myns=s033&mynp=OCSTXKQY&mynp=OCSWJ00&mync=E&cm_sp=s033-_-OCSTXKQY-OCSWJ00-_-E" rel="noreferrer" target="_blank">http://www.ibm.com/support/docview.wss?uid=ssg1S1010233&myns=s033&mynp=OCSTXKQY&mynp=OCSWJ00&mync=E&cm_sp=s033-_-OCSTXKQY-OCSWJ00-_-E</a>><br>
> - ABSTRACT: IBM has identified an issue with all IBM GPFS and IBM<br>
> Spectrum Scale versions where the NSD server is enabled to use RDMA for<br>
> file IO and the storage used in your GPFS cluster accessed via NSD<br>
> servers (not fully SAN accessible) includes anything other than IBM<br>
> Elastic Storage Server (ESS) or GPFS Storage Server (GSS); under these<br>
> conditions, when the RDMA-enabled network adapter fails, the issue may<br>
> result in undetected data corruption for file write or read operations.<br>
><br>
> ------------------------------------------------------------------------------<br>
> Manage your My Notifications subscriptions, or send questions and<br>
> comments.<br>
> - Subscribe or Unsubscribe -<br>
> <a href="https://www.ibm.com/support/mynotifications" rel="noreferrer" target="_blank">https://www.ibm.com/support/mynotifications</a><br>
> <<a href="https://www.ibm.com/support/mynotifications" rel="noreferrer" target="_blank">https://www.ibm.com/support/mynotifications</a>><br>
> - Feedback -<br>
> <a href="https://www-01.ibm.com/support/feedback/techFeedbackCardContentMyNotifications.html" rel="noreferrer" target="_blank">https://www-01.ibm.com/support/feedback/techFeedbackCardContentMyNotifications.html</a><br>
> <<a href="https://www-01.ibm.com/support/feedback/techFeedbackCardContentMyNotifications.html" rel="noreferrer" target="_blank">https://www-01.ibm.com/support/feedback/techFeedbackCardContentMyNotifications.html</a>><br>
><br>
> - Follow us on Twitter - <a href="https://twitter.com/IBMStorageSupt" rel="noreferrer" target="_blank">https://twitter.com/IBMStorageSupt</a><br>
> <<a href="https://twitter.com/IBMStorageSupt" rel="noreferrer" target="_blank">https://twitter.com/IBMStorageSupt</a>><br>
><br>
><br>
><br>
> To ensure proper delivery please add <a href="mailto:mynotify@stg.events.ihost.com" target="_blank">mynotify@stg.events.ihost.com</a><br>
> <mailto:<a href="mailto:mynotify@stg.events.ihost.com" target="_blank">mynotify@stg.events.ihost.com</a>> to<br>
> your address book.<br>
> You received this email because you are subscribed to IBM My<br>
> Notifications as:<br>
> <a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a> <mailto:<a href="mailto:aaron.s.knister@nasa.gov" target="_blank">aaron.s.knister@nasa.gov</a>><br>
><br>
> Please do not reply to this message as it is generated by an automated<br>
> service machine.<br>
><br>
> (C) International Business Machines Corporation 2017. All rights<br>
> reserved.<br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a> <<a href="http://spectrumscale.org" rel="noreferrer" target="_blank">http://spectrumscale.org</a>><br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
> <<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>><br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a> <<a href="http://spectrumscale.org" rel="noreferrer" target="_blank">http://spectrumscale.org</a>><br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
> <<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
><br>
<br>
--<br>
Aaron Knister<br>
NASA Center for Climate Simulation (Code 606.2)<br>
Goddard Space Flight Center<br>
<a href="tel:(301)%20286-2776" value="+13012862776" target="_blank">(301) 286-2776</a><br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div>