[gpfsug-discuss] Network switches/architecture for Spectrum Scale (GPFS)

Frank Kraemer kraemerf at de.ibm.com
Mon Mar 23 07:46:54 GMT 2020


Hi,

> May I know if could feedback from anyone who is using Arista or Mellanox
switches on the
> clusters to understand the pros and cons, stability and the performance
numbers of the same?

Most current network switches are using Merchant Silicon ASICs instead of
custom ASICs as in the past. (This is even true for some Cisco devices.)

"Merchant silicon is a term used to described chips, usually ASICs, that
are designed and made by an entity other than the company selling the
switches in which they are used. I might be tempted to say such switches
use off-the-shelf ASICs, though that might imply that I could buy these
chips from a retail store."

https://www.datacenterknowledge.com/networks/why-merchant-silicon-taking-over-data-center-network-market

All of the Merchant Silicon ASICs lines have different family members from
entry to high-end features. Switch vendors will pick and choose these ASICs
by price and target market for the specific switch models. You can't tell
from the "outside" which switch does offer which kind of performance. You
need to do more research on the specific switch and use case. Most of the
time Mellanox switches does offer very attractive features and performance
levels for parallel I/O workloads but there are more options in the market.

The NOS (Network OS) which is running on these new networking switches is
also not a fixed option. You have some choice here. Some switches can be
ordered with different NOS versions. Which NOS is the best fit for Spectrum
Scale or other pFS's is still a topic to be researched.

Last but not least the question of which NIC card is the best fit in this
job is important. SmartNIC: A network interface card (network adapter) that
offloads processing tasks that the system CPU would normally handle. Using
its own on-board processor, the SmartNIC may be able to perform any
combination of encryption/decryption, firewall, TCP/IP and HTTP processing.
SmartNICs can be ASIC, FPGA, and System-on-a-Chip (SOC) based. Naturally
vendors who make just one kind of NIC seem to insist that only the type of
NIC they make should qualify as a SmartNIC. Mellanox is a good choice here
but there is wide range of choices and features already in the market.

-frank-

Appendix:

1 - A Network-centric View of Scalable Storage by Andy Bechtolsheim, Chief
Development Officer and Co-Founder, Arista Networks
https://youtu.be/__LomgSguSc

2 - Broadcom Ships Tomahawk 4, Industry’s Highest Bandwidth Ethernet Switch
Chip at 25.6 Terabits per Second – Broadcom

https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56990-series
https://packetpushers.net/podcast/network-break-264-broadcoms-new-tomahawk-4-hits-25-6tbps-juniper-announces-sd-lan-for-ex-switches/
https://www.telecompaper.com/news/broadcom-ships-tomahawk-4-ethernet-switch-chip--1319338

Broadcom delivered the StrataXGS Tomahawk 4 switch series, demonstrating an
unprecedented 25.6 Tbps of Ethernet switching performance in a single
device.

3 - Mellanox Introduces Revolutionary ConnectX-6 Dx and BlueField-2 Secure
Cloud SmartNICs and I/O Processing Unit Solutions

ConnectX-6 Dx and BlueField-2 Provide 200 Gb/s Ethernet and InfiniBand
Connectivity, Enabling Next Generation of Clouds, Secure Data Centers and
Storage Platforms
https://www.mellanox.com/page/press_release_item?id=2195


Frank Kraemer
IBM Consulting IT Specialist  / Client Technical Architect
Am Weiher 24, 65451 Kelsterbach, Germany
mailto:kraemerf at de.ibm.com
Mobile +49171-3043699
IBM Germany
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200323/eda21b50/attachment-0001.htm>


More information about the gpfsug-discuss mailing list