SmartNIC Linux router for modern data center | Hacker News Reader

SmartNIC Linux router for modern data center

netris.ai

41 pointsalex_saroyan13 comments

Wow, such a nicely written and informative article, and why this is not on the front page is beyond me.

Personally, recently I've been lamenting about the lack of popular Linux distro for network operating system (NOS) [1]. I really wish we have similar eco-system for Linux networking as for server and mobile (if we can count Android as Linux). Next time If anyone ask me why we want that I'll just point to this article, thanks Alex. The SDN trends in networking in particular NFV/NVF really make Linux and networking a match made in heaven.

Just wondering is SmartNIC necessary for the typical private cloud linux router set up? How about eBPF enabled routing to bypass the kernel, is it good enough performance wise for small to medium data center?

[1]https://news.ycombinator.com/item?id=29959234

Thanks.

Our customers are operating private cloud deployments, running Linux on network switches, and on their SoftGate nodes.

Linux made its place to the data center networking. We need to spread this knowledge further to make Linux Networking mainstream.

eBPF is awesome. Container networking inside the server is the best place for eBPF networking.

For data center border networking (border router, full routing table, Layer-4 load balancing, NAT, VPN) DPDK enables us to push the boundaries of Linux Networking, squeezing 100Gbps / 30Mpps out of one server. It's performance comparable with Cisco ASR and Juniper MX specialized hardware.

The Linux router + Linux switches, wrapped into Netris automatic NetOps software enables users to operate their on-prem network with cloud-like experience. Plugging in their favorite tools like Kubernetes, Terraform, etc.

What kind of protocol offload capabilities it has ?

We leverage DPDK & SmartNIC to accelerate Layer-3 routing to 100Gbps / 30Mpps with 1M routes in the routing table.

This approach allows for border router, NAT, L4LB functionality with performance comparable to Cisco ASR / Juniper MX routers.

Can you use it for deep packet inspection process?

You can hook a DPI into Netris operated network, yes.

DPDK Lib is good when you want to have user space app working with the IO bypassing the kernel. however it may greatly harm the system performance in the cases when kernel thread will hook the CPU resources and throw away you user space thread. This can introduce milliseconds of latency in your routing path. Also one question do you think to support L3 PFC support ? This is very important in the infrastructures where number of nodes > 1K.

Right. We instruct Kernel to reserve 8 CPU cores exclusively for our user space application that is responsible for traffic forwarding.

Here's performance results. Max latency is 0.31 msec.

####################################################

min_latency_ns: 708

avg_latency_ns: 63723

max_latency_ns: 310795

jitter_ns: 30940

####################################################

Regarding L3 PFC support. In my understanding it's a ROCE requirement. I view RoCE as east-west traffic, since its not leaving the boundaries of the switch fabric, I think non-blocking leaf/spine fabric should be sufficient there. Please correct me if'm missing something.

SoftGate, the Linux router where we use DPDK, is designed for North/South traffic, sitting on the border of the data center, so we don't expect RoCE traffic there, but again, please let me know if you see it differently.

You have got a very good latency numbers. It's impressive.

You are right about RoCE traffic. It is mainly East-West traffic. However NTT has a project called IOWN and in the scope of that project as I know they are trying to build North-South traffic based on the RoCE technology. In the case when you will decide to use your SoftGate routers inside the datacenter it would make sense to have a multipath routing capabilities and also solution for "PFC spreading blocking problem". This problem is not particularly bound with RoCE transport layer but instead for a MPI use cases. A several big infrastructure companies built their own network transport layer and in house hardware to solve those problems.

What industries are the current use cases from?

We commonly see, online entertainment providers, online ad providers, AI/ML companies, Edge & private cloud builders.

Every company that builds a private cloud or edge can easily adopt this technology. It's designed for cloud practitioner, DevOps, and NetOps engineers in mind.

What is different in your product?

100Gbps / 30Mpps - border router, on $5000 Linux server and decent SmartNIC. (Alternative to $20-200K Cisco ASR/Juniper MX routers)

Native integration with Kubernetes and Terraform, designed for modern cloud practitioners, DevOps, and NetOps folks.

Cloud-like, self-operating, functionality.

Reply on news.ycombinator.com