[gpfsug-discuss] Services on DSS/ESS nodes

Mon Oct 5 07:27:39 BST 2020

Coming to the routing point, is there any reason why you need it ? I mean, this is because GPFS trying to connect between compute nodes or a reason outside GPFS scope ?
If the reason is GPFS,  imho best approach - without knowledge of the licensing you have - would be to use separate clusters: a storage cluster and two compute clusters.

Both compute clusters join using multicluster setup the storage cluster. There is no need both compute clusters see each other, they only need to see the storage cluster. One of the clusters using the 10G, the other cluster using the IPoIB interface.
You need at least three quorum nodes in each compute cluster but if licensing is per drive on the DSS, it is covered.
--
Jordi Caubet Serrabou
IBM Software Defined Infrastructure (SDI) and Flash Technical Sales Specialist
Technical Computing and HPC IT Specialist and Architect
Ext. Phone: (+34) 679.79.17.84 (internal 55834)
E-mail: jordi.caubet at es.ibm.com

> On 5 Oct 2020, at 08:19, Olaf Weiser <olaf.weiser at de.ibm.com> wrote:
> 
> 
> let me add a few comments from some very successful large installations in Eruope
>  
> # InterOP
> Even though (as Luis pointed to) , there is no support statement to run intermix DSS/ESS in general, it was ~, and is, and will be, ~ allowed for short term purposes, such as e.g migration.
> The reason to not support those DSS/ESS mixed configuration in general is simply driven by the fact, that different release version of DSS/ESS potentially (not in every release, but sometimes)  comes with different driver levels, (e.g. MOFED), OS, RDMA-settings, GPFS tuning,  etc...
> Those changes can have an impact/multiple impacts and therefore, we do not support that in general. Of course -and this would be the advice for every one - if you are faced the need to run a mixed configuration for e.g. a migration and/or e.g. cause of you need to temporary provide space etc... contact you IBM representative and settle to plan that accordingly..
> There will be (likely) some additional requirements/dependencies defined  like  driver versions, OS,  and/or Scale versions, but you'll get a chance to run mixed configuration - temporary limited to your specific scenario.
>  
> # Monitoring
> No doubt, monitoring is essential and absolutely needed. - and/but - IBM wants customers to be very sensitive, what kind of additional software (=workload) gets installed on the ESS-IO servers. BTW, this rule applies as well to any other important GPFS node with special roles (e.g. any other NSD server etc)
> But given the fact, that customer's usually manage and monitor their server farms from a central point of control (any 3rd party software), it is common/ best practice , that additionally monitor software(clients/endpoints) has to run on GPFS nodes, so as on ESS nodes too.
>  
> If that way of acceptance applies for DSS too, you may want to double check with Lenovo ?!
>  
>  
> #additionally GW functions
> It would be a hot iron, to general allow routing on IO nodes. Similar to the mixed support approach, the field variety for such a statement would be hard(==impossible) to manage. As we all agree, additional network traffic can (and in fact will) impact GPFS.
> In your special case, the expected data rates seems to me more than ok and acceptable to go with your suggested config (as long workloads remain on that level / monitor it accordingly as you are already obviously doing) 
> Again,to be on the safe side.. contact your IBM representative and I'm sure you 'll find a way..
>  
>  
>  
> kind regards....
> olaf
>  
>  
> ----- Original message -----
> From: Jonathan Buzzard <jonathan.buzzard at strath.ac.uk>
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> To: gpfsug-discuss at spectrumscale.org
> Cc:
> Subject: [EXTERNAL] Re: [gpfsug-discuss] Services on DSS/ESS nodes
> Date: Sun, Oct 4, 2020 12:17 PM
>  
> On 04/10/2020 10:29, Luis Bolinches wrote:
> > Hi
> >
> > As stated on the same link you can do remote mounts from each other and
> > be a supported setup.
> >
> > “ You can use the remote mount feature of IBM Spectrum Scale to share
> > file system data across clusters.”
> >
> 
> You can, but imagine I have a DSS-G cluster, with 2PB of storage on it
> which is quite modest in 2020. It is now end of life and for whatever
> reason I decide I want to move to ESS instead.
> 
> What any sane storage admin want to do at this stage is set the ESS, add
> the ESS nodes to the existing cluster on the DSS-G then do a bit of
> mmadddisk/mmdeldisk and sit back while the data is seemlessly moved from
> the DSS-G to the ESS. Admittedly this might take a while :-)
> 
> Then once all the data is moved a bit of mmdelnode and bingo the storage
> has been migrated from DSS-G to ESS with zero downtime.
> 
> As that is not allowed for what I presume are commercial reasons (you
> could do it in reverse and presumable that is what IBM don't want) then
> once you are down the rabbit hole of one type of storage the you are not
> going to switch to a different one.
> 
> You need to look at it from the perspective of the users. They frankly
> could not give a monkeys what storage solution you are using. All they
> care about is having usable storage and large amounts of downtime to
> switch from one storage type to another is not really acceptable.
> 
> 
> JAB.
> 
> --
> Jonathan A. Buzzard                         Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss 
>  
>  
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> 

Salvo indicado de otro modo más arriba / Unless stated otherwise above:
International Business Machines, S.A.
Santa Hortensia, 26-28, 28002 Madrid
Registro Mercantil de Madrid; Folio 1; Tomo 1525; Hoja M-28146
CIF A28-010791

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201005/1846e9d6/attachment-0002.htm>