<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">The point of the original question was to discover why there is a warning about performance for nsdChksumTraditional=yes, but that warning doesn’t seem to apply to an ESS environment.<div class=""><br class=""></div><div class="">Your reply was that checksums in an ESS environment are calculated in parallel on the NSD server based on the physical storage layout used underneath the NSD, and is thus faster. My point was that if there is never a checksum calculated by the NSD client, then how does the NSD server know that it got uncorrupted data?<div class=""><br class=""></div><div class="">The link you referenced below (thank you!) indicates that, in fact, the NSD client DOES calculate a checksum and forward it with the data to the NSD server. The server validates the data (necessitating a re-calculation of the checksum), and then GNR stores the data, A CHECKSUM[1], and some block metadata to media. </div><div class=""><br class=""></div><div class="">So this leaves us with a checksum calculated by the client and then validated (re-calculated) by the server — IN BOTH CASES. For the GNR case, another checksum in calculated and stored with the data for another purpose, but that means that the nsdChksumTraditional=yes case is exactly like the first phase of the GNR case. So why is that case slower when it does less work? Slow enough to merit a warning, no less!</div><div class=""><br class=""></div><div class="">I’m really not trying to be a pest, but I have a logic problem with either the question or the answer — they aren’t consistent (or I can’t rationalize them to be so).</div><div class=""><br class=""></div><div class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">-- <br class="">Stephen<br class=""><br class=""><div class="">[1] The document is vague (I believe intentionally, because it could have easily been made clear) as to whether this is the same checksum or a different one. Presumably the server-side-new-checksum is calculated in parallel and protects the chunklets or whatever they're called. This is all consistent with what you said!</div><div class=""><br class=""></div></div>
</div><div class=""><br class="webkit-block-placeholder"></div>
<div><br class=""><blockquote type="cite" class=""><div class="">On Oct 29, 2018, at 5:29 PM, Kumaran Rajaram <<a href="mailto:kums@us.ibm.com" class="">kums@us.ibm.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><font size="2" face="Arial" class="">In non-GNR setup, nsdCksumTraditional=yes enables
data-integrity checking between a traditional NSD client node and its NSD
server, at the network level only.</font><br class=""><br class=""><font size="2" face="Arial" class="">The ESS storage supports end-to-end checksum,
NSD client to the ESS IO servers (at the network level) as well as from
ESS IO servers to the disk/storage. This is further detailed
in the docs (link below):</font><br class=""><br class=""><a href="https://www.ibm.com/support/knowledgecenter/en/SSYSP8_5.3.1/com.ibm.spectrum.scale.raid.v5r01.adm.doc/bl1adv_introe2echecksum.htm" class=""><font size="2" color="blue" face="Arial" class="">https://www.ibm.com/support/knowledgecenter/en/SSYSP8_5.3.1/com.ibm.spectrum.scale.raid.v5r01.adm.doc/bl1adv_introe2echecksum.htm</font></a><br class=""><br class=""><font size="2" face="Arial" class="">Best,</font><br class=""><font size="2" face="Arial" class="">-Kums</font><br class=""><br class=""><br class=""><br class=""><br class=""><br class=""><font size="1" color="#5f5f5f" face="sans-serif" class="">From:
</font><font size="1" face="sans-serif" class="">Stephen Ulmer <<a href="mailto:ulmer@ulmer.org" class="">ulmer@ulmer.org</a>></font><br class=""><font size="1" color="#5f5f5f" face="sans-serif" class="">To:
</font><font size="1" face="sans-serif" class="">gpfsug main discussion
list <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="">gpfsug-discuss@spectrumscale.org</a>></font><br class=""><font size="1" color="#5f5f5f" face="sans-serif" class="">Date:
</font><font size="1" face="sans-serif" class="">10/29/2018 04:52 PM</font><br class=""><font size="1" color="#5f5f5f" face="sans-serif" class="">Subject:
</font><font size="1" face="sans-serif" class="">Re: [gpfsug-discuss]
NSD network checksums (nsdCksumTraditional)</font><br class=""><font size="1" color="#5f5f5f" face="sans-serif" class="">Sent by:
</font><font size="1" face="sans-serif" class=""><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a></font><br class=""><hr noshade="" class=""><br class=""><br class=""><br class=""><font size="3" class="">So the ESS checksums that are highly touted as "protecting
all the way to the disk surface" completely ignore the transfer between
the client and the NSD server? It sounds like you are saying that all of
the checksumming done for GNR is internal to GNR and only protects against
bit-flips on the disk (and in staging buffers, etc.)</font><br class=""><br class=""><font size="3" class="">I’m asking because your explanation completely ignores
calculating anything on the NSD client and implies that the client could
not participate, given that it does not know about the structure of the
vdisks under the NSD — but that has to be a performance factor for both
types if the transfer is protected starting at the client — which it is
in the case of nsdCksumTraditional which is what we are comparing to ESS
checksumming.</font><br class=""><br class=""><font size="3" class="">If ESS checksumming doesn’t protect on the wire I’d
say that marketing has run amok, because that has *definitely* been implied
in meetings for which I’ve been present. In fact, when asked if Spectrum
Scale provides checksumming for data in-flight, IBM sales has used it as
an ESS up-sell opportunity.</font><br class=""><br class=""><font size="3" class="">-- </font><br class=""><font size="3" class="">Stephen<br class=""><br class=""></font><br class=""><br class=""><font size="3" class="">On Oct 29, 2018, at 3:56 PM, Kumaran Rajaram <</font><a href="mailto:kums@us.ibm.com" class=""><font size="3" color="blue" class=""><u class="">kums@us.ibm.com</u></font></a><font size="3" class="">>
wrote:</font><br class=""><br class=""><font size="2" face="Arial" class="">Hi,</font><font size="3" class=""><br class=""></font><font size="2" face="Arial" class=""><br class="">>>How can it be that the I/O performance degradation warning only
seems to accompany the nsdCksumTraditional setting and not GNR?<br class="">>>Why is there such a penalty for "traditional" environments?</font><font size="3" class=""><br class=""></font><font size="2" face="Arial" class=""><br class="">In GNR IO/NSD servers (ESS IO nodes), the checksums are computed in parallel
for a NSD (storage volume/vdisk) across the threads handling each
pdisk/drive (that constitutes the vdisk/volume). This is possible since
the GNR software on the ESS IO servers is tightly integrated with underlying
storage and is aware of the vdisk DRAID configuration (strip-size, pdisk
constituting vdisk etc.) to perform parallel checksum operations. </font><font size="3" class=""><br class=""></font><font size="2" face="Arial" class=""><br class="">In non-GNR + external storage model, the GPFS software on the NSD server(s)
does not manage the underlying storage volume (this is done by storage
RAID controllers) and the checksum is computed serially. This would
contribute to increase in CPU usage and I/O performance degradation (depending
on I/O access patterns, I/O load etc).</font><font size="3" class=""><br class=""></font><font size="2" face="Arial" class=""><br class="">My two cents.</font><font size="3" class=""><br class=""></font><font size="2" face="Arial" class=""><br class="">Regards,<br class="">-Kums</font><font size="3" class=""><br class=""><br class=""><br class=""><br class=""><br class=""></font><font size="1" color="#5f5f5f" face="sans-serif" class=""><br class="">From: </font><font size="1" face="sans-serif" class="">Aaron
Knister <</font><a href="mailto:aaron.s.knister@nasa.gov" class=""><font size="1" color="blue" face="sans-serif" class=""><u class="">aaron.s.knister@nasa.gov</u></font></a><font size="1" face="sans-serif" class="">></font><font size="1" color="#5f5f5f" face="sans-serif" class=""><br class="">To: </font><font size="1" face="sans-serif" class="">gpfsug
main discussion list <</font><a href="mailto:gpfsug-discuss@spectrumscale.org" class=""><font size="1" color="blue" face="sans-serif" class=""><u class="">gpfsug-discuss@spectrumscale.org</u></font></a><font size="1" face="sans-serif" class="">></font><font size="1" color="#5f5f5f" face="sans-serif" class=""><br class="">Date: </font><font size="1" face="sans-serif" class="">10/29/2018
12:34 PM</font><font size="1" color="#5f5f5f" face="sans-serif" class=""><br class="">Subject: </font><font size="1" face="sans-serif" class="">[gpfsug-discuss]
NSD network checksums (nsdCksumTraditional)</font><font size="1" color="#5f5f5f" face="sans-serif" class=""><br class="">Sent by: </font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class=""><font size="1" color="blue" face="sans-serif" class=""><u class="">gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size="3" class=""><br class=""></font><hr noshade="" class=""><font size="3" class=""><br class=""><br class=""></font><tt class=""><font size="2" class=""><br class="">Flipping through the slides from the recent SSUG meeting I noticed that
<br class="">in 5.0.2 one of the features mentioned was the nsdCksumTraditional flag.
<br class="">Reading up on it it seems as though it comes with a warning about <br class="">significant I/O performance degradation and increase in CPU usage. I <br class="">also recall that data integrity checking is performed by default with <br class="">GNR. How can it be that the I/O performance degradation warning only <br class="">seems to accompany the nsdCksumTraditional setting and not GNR? As <br class="">someone who knows exactly 0 of the implementation details, I'm just <br class="">naively assuming that the checksum are being generated (in the same <br class="">way?) in both cases and transferred to the NSD server. Why is there such
<br class="">a penalty for "traditional" environments?<br class=""><br class="">-Aaron<br class=""><br class="">-- <br class="">Aaron Knister<br class="">NASA Center for Climate Simulation (Code 606.2)<br class="">Goddard Space Flight Center<br class="">(301) 286-2776<br class="">_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at </font></tt><a href="http://spectrumscale.org/" class=""><tt class=""><font size="2" color="blue" class=""><u class="">spectrumscale.org</u></font></tt></a><font size="3" color="blue" class=""><u class=""><br class=""></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class=""><tt class=""><font size="2" color="blue" class=""><u class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></tt></a><tt class=""><font size="2" class=""><br class=""></font></tt><font size="3" class=""><br class=""><br class=""><br class="">_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at </font><a href="http://spectrumscale.org/" class=""><font size="3" color="blue" class=""><u class="">spectrumscale.org</u></font></a><font size="3" color="blue" class=""><u class=""><br class=""></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class=""><font size="3" color="blue" class=""><u class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><br class=""><tt class=""><font size="2" class="">_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class=""><tt class=""><font size="2" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt class=""><font size="2" class=""><br class=""></font></tt><br class=""><br class="">
_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class=""></div></blockquote></div><br class=""></div></div></body></html>