HNHacker News
TopNewBestAskShowJobs

dtaht

555 karma · joined June 1, 2014

submissionscomments
dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
I have just taken the time to point out a few things he did more quietly, later in life, with enormous impact.
dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
There is an even sadder backstory to the whole LF thing, and the time it got created and sucked the air out of the room. I wish we had found a way to get this idea off the ground, and established the academic and early response institution we dreamed of.

https://web.archive.org/web/20141217131407/http://icei.org/

But I do not want to talk about it, or how I feel about LF, the loss of GPL enforcement, the state of the embedded market, or the demise of linux journal, and of so many other ideals I still hold dear. I just keep plugging on, fixing bufferbloat, and home routers. Y'all caught me on a friday night, after a long week, trying to improve the cybersecurity label initiative at the FCC, as well as girding loins for yet another pointless battle over pointless points over network neutrality debate.

dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
One of my great sadnesses about the long term support of FOSS was that the great redhat experiment of letting open source contributors into the IPO´s strike price, has never been repeated. For a tiny amount of re-investment into the commons like this, it would have been a better, far less scary and undermaintained world.

I really felt then, that misunderstood, and underappreciated, complicated, dedicated creators of really great code, would have a floor under them from all the companies that used it and did not understand it.

https://www.cnet.com/tech/tech-industry/late-ipo-change-left...

RHAT going public and sharing the IPO strike prices with the devs they based their success on, was a great day for the prospect of an indirect financial return on years of craftsmanship and care.

I hope one day, the idea finds new life.

dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
The core infrastructure initiative sucked a lot of air into their room, and accomplished almost nothing.
dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
I strongly disagree that esr´s books are irrelevant. You had to be there, in 1998, I guess, facing the prospect of being fired for working on free software, as many were, or facing down the might of the legal apparatus of microsoft, and admiring his rhetorical skill at facing down the suits, as many did, then. He traveled all over the world, giving many inspirational talks.

He books were highly influential on me, and if it were not for his encouragement, and support, there would not be, oh, 10B copies of fq_codel running today, cake, or sqm, or better wifi routers, or libreqos.

We had fun, too!

http://esr.ibiblio.org/?p=4162

I did learn from his lesson - when he told me I had to be a front man for the bufferbloat effort, in 2013, for at least 5 years - he quoted the original of this:

“First they ignore you, then they laugh at you, then they fight you, then you win.” at me... and muttered under his breath,

"... and then you become a pariah."

I learned from his example.

So far, by staying on topic, and keeping my political and personal opinions out of any engagement with anyone on the web, and not writing controversial blogs (I know he has at least 3 he regrets)... I have largely escaped being a pariah, but, have also been working on fixing the same darn thing, for 7 years longer than I ever wanted to.

dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
I should totally take that down. Nobody cared. I got, oh, perhaps 70 dollars a month more into my patreon via this idea...

... but I like to think, that the awareness of the problem itself has propagated all the way up to the highest levels of many governments.

dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
I just heard about that recently! I am encouraged! I hope they put some dough under OpenWrt. NLNET has also been a consistent and easy to apply to funder.
dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
He never cashed out. He got nothing. Just the royalties on his books and the occasional consulting contract.
dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
His idea was published in the last linux journal, and generated in part, in sympathy for my state of health at the time. Since then there has been a lot of decent research on the "load" and emerging government support for critical projects, as log4j woke up quite a few folk as to the problems that we are creating for ourselves with bad maintenence of open source code. The FCC has a NPRM pending on a cybersecurity label, for example, based on NISTś recommendations from this presidential directive.

https://www.whitehouse.gov/briefing-room/statements-releases...

It is ok that the loadsharers idea, as conceived here, has died, as it lives on in other places, and the things that esr and I cared about then, are getting taken care of.

dtaht··on Loadsharers: Funding the Load Bearing Internet People (2020)
I am surprised to see this show up again. The idea was ahead of its time, and sank without a trace. Since then, (and during) multiple orgs have attempted to also put some sort of floor below various maintainers. I see these finally beginning to succeed in some places, although I still struggle on a combination of patreon donations and the occasional grant, for what I do. Simon is semi-retired now but I think dnsmasq is mostly out of trouble. I can think of many others still struggling in obscurity.

Eric himself, had a bout with stomach cancer, and has mostly withdrawn from the web, tired of the perpetual onslaught and criticism. He´s been working on a new book, for a very long time. IMHO he was the right person at the right time, to get open source into the mainstream, and although many might disagree with him, and his politics, I think his contribution to internet history is assured.

ESR´s most recent project was ntpsec.org, which vastly reduced the attack surface of ntp. Another key one used in many places, and needed less now, was reposurgeon: http://www.catb.org/esr/reposurgeon/repository-editing.html

dtaht··on Net Neutrality´s origins in bufferbloat, BitTorrent vs. VoIP
Moving forward I hope policy decisions are made with an eye towards truly making the internet faster. https://blog.cloudflare.com/making-home-internet-faster/
dtaht··on Richard Stallman reveals he has cancer in the GNU 40 Hacker Meeting talk [video]
All the that is needed for evil to triumph is for good men to do nothing.
dtaht··on Net Neutrality´s origins in bufferbloat, BitTorrent vs. VoIP
That is a beautiful thread as to what was the original source of the network neutrality bittorrent vs voip bufferbloat blowup.

https://twitter.com/jlivingood/status/1707078242857849244

With tons of links to all the technical activity it sparked since.

Seeing all the political activity tied onto it since (and now again) reminds me of two families at war about an incident that had happened generations and generations before, where the two sides no longer remembered why they hated each other so, but just went on hating, and not forgiving, and not moving on.

Yes, there are entirely separate and additional NN issues, but the technical problem of providing common carriage between two very different network application types (voip/gaming vs file transfer) is thoroughly solved now, and if only all sides recognised at least this much, and made peace over it, and worked together to deploy those solutions, maybe, just maybe, we could find mutually satisfactory solutions to the other problems that plague the internet today, like security, and the ipv6 rollout.

If anyone here knows anyone more political, still vibrating with 10+ years of outrage about NN on this fronts, on one side or the other, if you could sit them down, over a beer, and try to explain that at the start it was a technical problem nobody understood at the time, maybe that would help

dtaht··on Net Neutrality´s origins in bufferbloat, BitTorrent vs. VoIP
If only more political folk understood how impossible it was in 2005 to achieve "common carriage" for two very different protocols - bulk torrents, vs VOIP - and how solved that problem is now, technically (fq_codel, CAKE, PIE, BBR, packet pacing), perhaps the resumption of the NN debate would shed more light, than heat, and actually speed adoption of these protocols.
dtaht··on TCP's congestion control saved the internet
Thank you for leveraging it.

but: Eric Dumazet came up with the flow queuing fast/slow queue concept in a single saturday afternoon in his hammock (after admittedly 9 months of hard work on every other fq and aqm algorithm), slammed it into the kernel the next day, and has been building ever more amazing things ever since.

I try to make sure people understand that I am more the acolyte of kathie, van, and Eric, than inventor here. I was proud of this thing we´d developed called SFQred, and writing a paper on it... but then kathie and van came out with codel a month later... obsoleting that part...

Eric and I implemented it over a week, saw that it was far better than RED... I went to sleep for a week, only to wake up to fq_codel, which solved everything.

We have been just re-implementing it and improving the edge cases since. I am pretty happy with cake, now. The world evolves however, and there are new demands on queuing. I cover some of these here: https://docs.google.com/document/d/1tTYBPeaRdCO9AGTGQCpoiuLO...

dtaht··on TCP's congestion control saved the internet
fq_codel was successfully offloaded in a few products, however, no offload here, just a xeon with a ton of cores and a big L3. On the libreqos case we leveraged some ebpf to do packet steering in what we call the heimdal bridge, and also kathie nichol´s wonderful passive ping concept: https://github.com/thebracket/cpumap-pping

src here: https://github.com/LibreQoE/LibreQoS/

You can get a transparent high speed bridge with shaping capability up pretty rapidly with supported hardware.

I still long for an ethernet card that can do a trie lookup natively! The flamegraphs are mostly getting the right packet to the right cpu, still.

dtaht··on TCP's congestion control saved the internet
fq+ we have been running at 25+gbit or more, for quite some time. There are typically 32k FQ_codels in a 10gbit system. I am pretty sure, that we would not have scaled past 10gbit as well as we did, without having made fq_codel and sch_fq (which does pacing), had we not. We run up to 10k instances of cake (4k queues each) in libreqos. Amazing results.
dtaht··on TCP's congestion control saved the internet
I hope more folk try this code out!!
dtaht··on TCP's congestion control saved the internet
Here is a realworld example, easily duplicated by anyone with a big fifo AND enough flows, that happens all the time, with cubic: https://blog.cerowrt.org/post/juniper/
dtaht··on TCP's congestion control saved the internet
This is a really good (and rare) interview with van, where he and mike finally asked the question - "how did this EVER work?": https://www.youtube.com/watch?v=QP4A6L7CEqA
dtaht··on TCP's congestion control saved the internet
exactly right. https://blog.cerowrt.org/post/juniper/

We have now produced variants of the flow queuing algorithm for fq-codel, fq-pie, and the FQ in cake, which we are now using in libreqos. It is the best form of fq ever designed, IMHO, with really good min/max fairness properties, resistence to starvation, great rtt fairness and highly complementary simultaneously time based AQM technology easily applied. I would like to see more improvements on the AQM side in the future and for it to land in more hardware.

Admittedly, I´m one of the authors... :)

dtaht··on TCP's congestion control saved the internet
Dumb enough to me, includes basic fq and aqm technology at the router. FIFOs are the worst form of queuing.

FQ solves most of the problem thoroughly enabling delay based transports, Van and Kathie´s codel, most of the rest. Van was also on the team that produced BBR, which spends most of it´s time in delay based mode when fq_codel is on the path.

I do wish more folk were actively applying cake nowadays. It works wonders.

https://blog.cerowrt.org/post/juniper/

Libreqos is doing nice things with cake.

dtaht··on TCP's congestion control saved the internet
The article kind of missed that Van was the author of RED, and later he and kathie nichols did codel, and later he was on the team that did BBR.

What would the internet have looked like without Van?

dtaht··on Some proposed improvements to fq_codel and CAKE
Over here I have been collecting feature requests and bug reports against the fq_codel and CAKE bufferbloat-fighting queuing disciplines, and (especially) seeking input from the field on problems observed and solutions found for things like 5G, Starlink, and WiFi.
dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
The embedded software industry in the US is nearly non-existent in the home router and IoT space. It is all leveraging ancient open source code, outsourced to the lowest bidder.
dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
This is not really relevant to the proposed rulemaking but it is something that bugs me deeply, and would like to get off my chest. I would like to see a mandate that red LED be wired inband to every camera and microphone, on every device, so if it is powered up, the LED is also. This is what John Gilmore proposed in 2004, and we adopted in the OLPC project, as the first step towards not being ubiquitously surveilled. It is low cost, low power, and easy to implement.
dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
This is a wonderful thread, and I am so glad to see so many sharing my nightmares and some of my conclusions. I encourage folk to work together on actually filing proposals with the FCC in their format. I would gladly join on such an effort, but am too busy to lead such an effort.
dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
Thank you very much for reading. It was the first, and last time, I ever took place in the public process and political action.

https://www.computerworld.com/article/2993112/vint-cerf-and-...

We were within months of delivering a massive RFC8290-based fix for wifi performance and we´d been bricking routers left and right... and then got in a whole bunch that we could not modify... due to that proposed regulation... I lost my temper, organized 260+ to sign, made that filing, won, and went back to work. I should perhaps have pressed harder, as the binary blob issue has got ever more terrifying, deeply embedded into too many baseband processors.

If I hold your attention a little bit? It would be rather nice if modern fq + aqm algorithms went into the internet nationwide. "Bufferbloat" is at epidemic proportions. Most the fixes arrived for it in Linux in 2012, and are only slowly rolling out 10+ years later, due to the accompanying epidemic of manufacturers´ not tracking new Linux kernels, (with all those accompanying vulnerabilities). I care very much about addressing security issues, but care about internet "latency under load" and better videoconferencing experiences more.

There´s only a billion or so devices left to upgrade.

https://www.usenix.org/system/files/conference/atc17/atc17-h...

dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
Agreed. I fully expect to see the routers we used in the cerowrt project from 2008 still operational for 10-20 more years. Thereś one out there with 4+ years of uptime that I know of.

https://blog.cerowrt.org/post/an_upgrade_in_place/

dtaht··on Ask HN: I’m an FCC Commissioner proposing regulation of IoT security updates
We have worked very hard in the OpenWrt and Linux projects to make it easy to update them in the field. Linux distros, android, apple, openwrt, etc have this facility built in now. IoT should also.
← PreviousPage 4 of 8Next →