Hacker News Reader

Comment by gmazza | Hacker News Reader

> Wrong: you see 10KB/s download speed because you are not throttling the incoming packets but the outgoing packets!

Yep. TC's default is to policy outgoing traffic, which in OP's example is a bunch of TCP ACKs essentially. Instead, they should be using ingress keyword, something like described here:

http://blog.stevedoria.net/20050906/ingress-policing-with-li...

Caveat emptor: ingress rate-limiting is hard. Long story short, it all boils down to what you do with non-confirming packets: There are two alternatives, and both are rather sub-optimal. You can either buffer/delay packets in kernel space (default, which leads to bufferbload and memory waste), or drop (which author linked above opted for, which leads to excessive retransmits and bandwidth waste).

View on HN

The drop vs buffer decision is no harder for outgoing or incoming packets. In either case: if you're trying to simulate a different kind of network, do what that network does. If you're just trying to get good QoS on your gateway router, then use a smart AQM that will buffer only to the extent that is reasonable, and then drop or ECN mark when buffering threatens to add too much latency.

What actually happens when Internet traffic makes the jump from my ISPs big-pipe backbone connection to my much slower last-mile connection? They clearly can't cache the world, and "excessive retransmits and bandwidth waste" doesn't seem to be the case?

They drop your packets, both for ingoing and outgoing overflow.

But how is that different from the proposed method of rate limited which causes "excessive retransmits and bandwidth waste"?

Not being pedantic, genuinely trying to understand how this stuff works.

Middle ground between dropping [1] and buffering is Active Queue Management as wtallis pointed out above. State of the art AQM on Linux nowadays is CoDel [2].

Big network gear vendors that ship to ISPs have adopted early forms of AQM (both RED [3] and proprietary algorithms) quite a while ago - they had to:

(a) backbone routers have much smaller buffer-to-bandwidth ratio (compared to a PC or even home router), so endless buffering is not an option;

(b) buffer tail drops (i.e. what drop keyword does when added to tc filters/disciplines) interact really poorly with TCP bandwidth control algorithms, and rather badly with RTP streams, too (so drop is not an option either - it ruins user's connections; and

(c) ISPs and carriers would typically run links (on average) much closer to saturation than your typical home router, so situation where router has to make buffer/drop decision is much, much more frequent.

[1] My point above was that you don't really want to drop packets on the receiver side, after that packet already traversed expensive part of the network.

[2] https://en.wikipedia.org/wiki/CoDel

[3] https://en.wikipedia.org/wiki/Random_early_detection

It's common to find that the buffers in front of the bottleneck last-mile link are sized for the highest tier of service regardless of what speed you're actually subscribed for. DSLReports has the only browser-based speed test that measures bufferbloat, and they have extensive data from that testing: https://www.dslreports.com/speedtest/results/bufferbloat?up=...

Your modem and the box on the other end of that bottleneck link are probably buffering far more than is reasonable. There's simply no reason for a cable modem to ever have in excess of 1s worth of backlog.

Reply on news.ycombinator.com