<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">One additional point to consider is what happens on a hardware failure.<div>eg. If you have two NSD servers that are both CES servers and one fails, then there is a double-failure at exactly the same point in time.<br><br><div id="AppleMailSignature"><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);">Daniel</span></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);"><img alt="/spectrum_storage-banne" src="http://ausgsa.ibm.com/projects/t/tivoli_visual_design/public/2015/Spectrum-Storage/Email-signatures/Storage/spectrum_storage-banner.png" width="432.5" height="auto" style="width: 601px; height: 5px;"></span></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><span style="background-color: rgba(255, 255, 255, 0);"><br> </span></div><table border="0" cellpadding="0" cellspacing="0" style="font-family: sans-serif;"><tbody><tr style="word-wrap: break-word; max-width: 447.5px;"><td style="word-wrap: break-word; max-width: 447.5px; width: 201px; padding: 0cm;"><div style="margin-bottom: 0.0001pt; line-height: normal;"></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><br></div><div style="margin-bottom: 0.0001pt; line-height: normal;"><table align="left" border="0" cellpadding="0" cellspacing="0" style="-webkit-text-size-adjust: auto;"><tbody><tr><td style="width: 130px; padding: 0cm;"><div style="margin-bottom: 0.0001pt; line-height: normal;"><a href="https://www.youracclaim.com/user/danel-kidger" target="_blank" style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"><font color="#000000" face=".SFUIDisplay"><img alt="IBM Storage Professional Badge" src="cid:" style="width: 120px; height: 120px; float: left;"></font></a></div></td><td style="width: 140px; padding: 0cm;"><a href="https://www.youracclaim.com/user/danel-kidger" style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"><font color="#000000" face=".SFUIDisplay"><img alt="" src="https://www.ibm.com/volunteers/images/resources/IBM_Volunteers_Silver_v6.png" style="width: 120px; height: 120px;"></font></a></td><td style="width: 0px; padding: 0cm;"><font face=".SFUIDisplay"><span style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"> </span></font></td><td style="width: 0px; padding: 0cm;"><font face=".SFUIDisplay"><span style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"> </span></font></td><td style="width: 0px; padding: 0cm;"><font face=".SFUIDisplay"><span style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"> </span></font></td><td style="width: 240px; padding: 0cm;"><div style="margin-bottom: 0.0001pt; line-height: normal;"><font face=".SFUIDisplay"><span style="-webkit-text-size-adjust: none; background-color: rgba(255, 255, 255, 0);"><strong>Dr Daniel Kidger</strong><br>IBM Technical Sales Specialist<br>Software Defined Solution Sales<br><br><a href="tel:+44-7818%20522%20266" dir="ltr" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1">+</a><a href="tel:+44-7818%20522%20266" dir="ltr" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1">44-(0)7818 522 266</a> <br><a href="mailto:daniel.kidger@uk.ibm.com" dir="ltr" x-apple-data-detectors="true" x-apple-data-detectors-type="link" x-apple-data-detectors-result="2">daniel.kidger@uk.ibm.com</a></span></font></div></td></tr></tbody></table></div></td><td style="word-wrap: break-word; max-width: 447.5px; width: 21px; padding: 0cm;"><font face=".SFUIDisplay"><span style="background-color: rgba(255, 255, 255, 0);"> </span></font></td><td style="word-wrap: break-word; max-width: 447.5px; width: 202px; padding: 0cm;"><br></td></tr></tbody></table><br></div><div><br>On 7 May 2018, at 16:39, Buterbaugh, Kevin L <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu">Kevin.Buterbaugh@Vanderbilt.Edu</a>> wrote:<br><br></div><blockquote type="cite"><div>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Hi All,
<div class=""><br class=""></div><div class="">I want to thank all of you who took the time to respond to this question … your thoughts / suggestions are much appreciated.</div><div class=""><br class=""></div><div class="">What I’m taking away from all of this is that it is OK to run CES on NSD servers as long as you are very careful in how you set things up. This would include:</div><div class=""><br class=""></div><div class="">1. Making sure you have enough CPU horsepower and using cgroups to limit how much CPU SMB and NFS can utilize.</div><div class="">2. Making sure you have enough RAM … 256 GB sounds like it should be “enough” when using SMB.</div><div class="">3. Making sure you have your network config properly set up. We would be able to provide three separate, dedicated 10 GbE links for GPFS daemon communication, GPFS multi-cluster link to our HPC cluster, and SMB / NFS communication.</div><div class="">4. Making sure you have good monitoring of all of the above in place.</div><div class=""><br class=""></div><div class="">Have I missed anything or does anyone have any additional thoughts? Thanks…</div><div class=""><br class=""></div><div class="">Kevin</div><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On May 4, 2018, at 11:26 AM, Sven Oehme <<a href="mailto:oehmes@gmail.com" class="">oehmes@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">there is nothing wrong with running CES on NSD Servers, in fact if all CES nodes have access to all LUN's of the filesystem thats the fastest possible configuration as you eliminate 1 network hop.
<div class="">the challenge is always to do the proper sizing, so you don't run out of CPU and memory on the nodes as you overlay functions. as long as you have good monitoring in place you are good. if you want to do the extra precaution, you could 'jail'
the SMB and NFS daemons into a c-group on the node, i probably wouldn't limit memory but CPU as this is the more critical resource to prevent expels and other time sensitive issues. </div><div class=""><div class=""><br class=""></div><div class="">sven<br class=""></div><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, May 4, 2018 at 8:39 AM Buterbaugh, Kevin L <<a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class="">Hi All,
<div class=""><br class=""></div><div class="">In doing some research, I have come across numerous places (IBM docs, DeveloperWorks posts, etc.) where it is stated that it is not recommended to run CES on NSD servers … but I’ve not found any detailed explanation of why not.</div><div class=""><br class=""></div><div class="">I understand that CES, especially if you enable SMB, can be a resource hog. But if I size the servers appropriately … say, late model boxes with 2 x 8 core CPU’s, 256 GB RAM, 10 GbE networking … is there any reason why I still should not combine
the two?</div><div class=""><br class=""></div><div class="">To answer the question of why I would want to … simple, server licenses.</div><div class=""><br class=""></div><div class="">Thanks…</div><div class=""><br class=""></div><div class="">Kevin</div><br class=""><div class=""><div class="">—</div><div class="">Kevin Buterbaugh - Senior System Administrator</div><div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div><div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" target="_blank" class="">Kevin.Buterbaugh@vanderbilt.edu</a> -
<a href="tel:(615)%20875-9633" value="+16158759633" target="_blank" class="">(615)875-9633</a></div><div class=""><br class=""></div><br class="m_7396076754734351969Apple-interchange-newline"></div><br class=""></div>_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C6ec06d262ea84752b1d408d5b1dbe2cc%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636610480314880560&sdata=WcjhER5Bbl8PsgvvDGYp%2BybpNcn1tB6ChyCaSI%2FrcJU%3D&reserved=0" rel="noreferrer" target="_blank" class="">spectrumscale.org</a><br class=""><a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C6ec06d262ea84752b1d408d5b1dbe2cc%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636610480314880560&sdata=J5%2F9X4dNeLrGKH%2BwmhIObVK%2BQ4oyoIa1vZ9F2yTU854%3D&reserved=0" rel="noreferrer" target="_blank" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class=""></blockquote></div></div></div></div>_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""><a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C6ec06d262ea84752b1d408d5b1dbe2cc%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636610480314880560&sdata=J5%2F9X4dNeLrGKH%2BwmhIObVK%2BQ4oyoIa1vZ9F2yTU854%3D&reserved=0" class="">https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C6ec06d262ea84752b1d408d5b1dbe2cc%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636610480314880560&sdata=J5%2F9X4dNeLrGKH%2BwmhIObVK%2BQ4oyoIa1vZ9F2yTU854%3D&reserved=0</a><br class=""></div></blockquote></div><br class=""></div>
</div></blockquote></div>Unless stated otherwise above:<BR>
IBM United Kingdom Limited - Registered in England and Wales with number 741598. <BR>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU<BR>
<BR>
</body></html>