I only know of Mellanox for their 10 gigabit ethernet cards. Does anybody know if this is a good or bad thing for Mellanox?
They also have some sort of a parallel VLIW CPU architecture that they've been trying to get off the ground for a while now called TILE/TILE64 so that might also play into things.
However since NVIDIA opened their offices in Israel a while ago they might simply be looking for an acquihire since Mellanox is a fabless semi chip maker it kinda fits that also.
Mellanox has driven IB speeds for more than a decade, limited only by PCIe bandwidth. Since they've had NICs that do both IB and Ethernet, they've been driving the ethernet market as well. We've been using their 100G adapters since 2015 (when they were first to market by a big margin). Even today, there are only a handful of vendors that can deliver a 100g NIC. I worry that if Mellanox stops driving port speed, we'll see a slower increase in the speed of NICs due to the lack of competition (eg, 400g will take longer..).
IB still has lower latency than Ethernet at least on paper especially when it comes to RDMA but I don't know how much of an issue that is for these applications.
But overall I'm not sure how much it matters to Mellanox since they are also the ones who are making the high speed Ethernet switches and host adapters.
You also need to use their cables and transceivers (or a similar alternative) for these speeds doesn’t matter if you are using Ethernet or IB.
I had a TileGX dev board and ported our product at the time (nearly 10 years ago). It was an ok arch but that’s a tough niche to fight for.
I’ve worked on several projects with them, and found they generally do a good job of feeding the beast when the OS and driver’s are properly tuned.
I’m not informed enough to call good or bad, but will instead say it’s interesting, especially in the HPC space (and the emerging AI space).
Mellanox has also embraced bringing RDMA to things like Ceph and working with the broader vendor ecosystem like Red Hat for using this in production.
I hope Nvidia doesn't taint the good reputation of this company.
They still are the company to go for infiniband, but infiniband it lost much of its appeal to non true supercomputing tasks.
Ethernet nowadays can do RDMA, soft guarantees on latency, in-order and reliable delivery at lower costs, and an option to reuse existing L2 networks. Mellanix has squeezed the infiniband cow dry.
And who did the Ethernet RDMA protocol (RoCEv1/v2) and sell the RDMA compatible NIC that everyone use in the HPC world currently ?
Tip: It is starting by Mella too.
I have no idea what it is, actually. Some ideas that may or may not matter (or might not even be correct):
- IB is a couple of decades younger, so could benefit from knowledge how to do fast protocols. (Not an explanation per se)
- Simpler forwarding. In IB the subnet manager gives out the LID's that are used for routing withing a subnet. They are shorter than an eth MAC (16 vs. 48 bits), so the lookups circuit in the switches can be smaller and faster(?), and also since the LID's are assigned by the subnet manager rather than being burned at the factory, they can be distributed taking into account the subnet topology, allowing switches to use LID Mask Count (LMC) filtering. Similarly, all routes within a subnet are calculated statically a priori by the subnet manager (load balancing among multiple paths is only static round robin, not dynamical load dependent), and don't have to be calculated on the fly by the switches.
- FEC rather than retransmission in case of corruption.
For everything else, RDMA on Ethernet buys you with ability to reuse your L2, and this matters way way more to people running DC businesses than anything else.